Data processing method and device

Abstract

The present application discloses a data processing method and device, and the specific implementation scheme is as follows: in response to receiving a page image, the page image is annotated to generate various image sets corresponding to the annotated data, each image set includes: a first image set for identifying the container type, a second image set for identifying text information, and a third image set for detecting image elements, and the page image is generated based on a page template; each image set is input into a trained image recognition model to generate a container type data set corresponding to the first image set, a text data set corresponding to the second image set, and an image element data set corresponding to the third image set; based on the template information of the page, the container type data set, the text data set, and the image element data set are converted to generate a template data set corresponding to the page image and upload it. The scheme uses image recognition technology to convert page images into template data, thereby achieving accurate positioning of template data.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of computer technology, specifically to the field of image recognition technology, and especially to a data processing method and device. Background Art

[0002] With the rapid development of the Internet, people are increasingly visiting various websites through web browsing, and thus the requirements for page construction are getting higher and higher. At present, the floor construction of dynamic pages generally adopts the template configuration method. Users select the template that meets the requirements in the template list area, and then customize the configuration style, data and other information to publish a complete online activity page. The template source can be a JSON (JavaScript Object Notation) file stored locally in the front-end project. Developers render floors according to the JSON string. Different templates need to create different files for template data storage. Summary of the invention

[0003] The present application provides a data processing method, apparatus, device and storage medium.

[0004] According to a first aspect of the present application, a data processing method is provided, the method comprising: in response to receiving a page image, annotating the page image, generating respective image sets corresponding to the annotated data, wherein the respective image sets comprise: a first image set for identifying a container type, a second image set for identifying text information, and a third image set for detecting image elements, and the page image is generated based on a page template; inputting the respective image sets into a trained image recognition model, generating a container type data set corresponding to the first image set, a text data set corresponding to the second image set, and an image element data set corresponding to the third image set, wherein the image recognition model is used to characterize the container type determination for each image in the first image set, the text detection and text recognition for each image in the second image set, and the image element detection and recognition for each image in the third image set; based on the template information of the page, converting the container type data set, the text data set, and the image element data set, generating a template data set corresponding to the page image, and uploading the template data set, wherein the conversion converts the container type data set, the text data set, and the image element data set based on a specific language structure.

[0005] In some embodiments, a page image is annotated to generate image sets corresponding to the annotated data, including: annotating the page image to obtain the annotated data corresponding to the page image; inputting the annotated data into a position determination model to generate position information of each block corresponding to the annotated data, wherein the position determination model is trained by historical related data of the annotated data; and determining each image set corresponding to the annotated data based on the position information of each block.

[0006] In some embodiments, the image recognition model is trained in the following manner: obtaining a training sample set, wherein the training samples in the training sample set include a first image set for identifying container types, a second image set for identifying text information, a third image set for detecting image elements, a container type data set corresponding to the first image set, a text data set corresponding to the second image set, and an image element data set corresponding to the third image set; using a deep learning method, taking the first image set, the second image set, and the third image set included in the training samples in the training sample set as input data, and taking the container type data set corresponding to the first image set, the text data set corresponding to the second image set, and the image element data set corresponding to the third image set as expected output data, to train the image recognition model.

[0007] In some embodiments, the image recognition model includes a container type recognition submodel, a text recognition submodel and an element recognition submodel; each image set is input into the trained image recognition model to generate a container type data set corresponding to the first image set, a text data set corresponding to the second image set and an image element data set corresponding to the third image set, including: inputting the first image set into the container type recognition submodel to generate a container type data set corresponding to the first image set, wherein the container type recognition submodel is used to characterize the container type determination for each image in the first image set; inputting the second image set into the text recognition submodel to generate a text data set corresponding to the second image set, wherein the text recognition submodel is used to characterize the text detection and text recognition for each image in the second image set; inputting the third image set into the element recognition submodel to generate an image element data set corresponding to the third image set, wherein the element recognition submodel is used to characterize the image element detection and recognition for each image in the third image set.

[0008] In some embodiments, the text recognition submodel includes a feature extraction submodel and a text sequence extraction submodel; the second image set is input into the text recognition submodel to generate a text data set corresponding to the second image set, including: inputting the second image set into the feature extraction submodel to obtain each feature matrix corresponding to the second image set, wherein the feature extraction submodel is constructed based on a convolutional neural network; inputting each feature matrix into the text sequence extraction submodel to obtain a text sequence corresponding to each feature matrix, wherein the text sequence extraction submodel is constructed based on a recursive neural network; based on each text sequence, determining the text information corresponding to each text sequence, and generating a text data set corresponding to each text information.

[0009] In some embodiments, the image recognition model and / or the container type recognition sub-model are built based on a deep residual network model.

[0010] In some embodiments, before converting the container type dataset, text dataset and image element dataset based on the template information of the page to generate a template dataset corresponding to the page image, it also includes: correcting the container type dataset, text dataset and image element dataset to obtain corrected container type dataset, text dataset and image element dataset, wherein the correction is used to characterize the analysis results based on the image position, image sequence and image repeatability of each image in each image set, and reordering the data in the container type dataset, text dataset and image element dataset.

[0011] In some embodiments, the correction is performed based on a combination of image scaling, image grayscale, image enhancement, image noise reduction, and image edge detection for each image in each image set.

[0012] In some embodiments, before correcting the container type dataset, text dataset and image element dataset to obtain the corrected container type dataset, text dataset and image element dataset, it also includes: performing content recognition on each image set to obtain a first dataset corresponding to the first image set, a second dataset corresponding to the second image set and a third dataset corresponding to the third image set; and correcting the data in the container type dataset, text dataset and image element dataset based on the comparison results of the first dataset, the second dataset and the third dataset with the container type dataset, text dataset and image element dataset to obtain the corrected container type dataset, text dataset and image element dataset.

[0013] In some embodiments, the method further includes: generating and displaying a template interface corresponding to the template data set based on the template data set; and / or optimizing a design scheme of the page template based on the template data set.

[0014] According to a second aspect of the present application, a data processing device is provided, the device comprising: a labeling unit, configured to label the page image in response to receiving a page image, and generate each image set corresponding to the labeling data, wherein each image set comprises: a first image set for identifying a container type, a second image set for identifying text information, and a third image set for detecting image elements, and the page image is generated based on a page template; a generation unit, configured to input each image set into a trained image recognition model, and generate a container type data set corresponding to the first image set, a text data set corresponding to the second image set, and an image element data set corresponding to the third image set, wherein the image recognition model is used to characterize the container type determination of each image in the first image set, the text detection and text recognition of each image in the second image set, and the image element detection and recognition of each image in the third image set; a conversion unit, configured to convert the container type data set, the text data set, and the image element data set based on the template information of the page, generate a template data set corresponding to the page image, and upload the template data set, wherein the conversion converts the container type data set, the text data set, and the image element data set based on a specific language structure.

[0015] In some embodiments, the labeling unit includes: a labeling module, configured to label the page image and obtain labeling data corresponding to the page image; a position generation module, configured to input the labeling data into a position determination model, and generate position information of each block corresponding to the labeling data, wherein the position determination model is trained by historical related data of the labeling data; and a determination module, configured to determine each image set corresponding to the labeling data based on the position information of each block.

[0016] In some embodiments, the image recognition model in the generation unit is trained using the following modules: an acquisition module, configured to acquire a training sample set, wherein the training samples in the training sample set include a first image set for identifying a container type, a second image set for identifying text information, a third image set for detecting image elements, a container type data set corresponding to the first image set, a text data set corresponding to the second image set, and an image element data set corresponding to the third image set; a training module, configured to use a deep learning method, using the first image set, the second image set, and the third image set included in the training samples in the training sample set as input data, and using the container type data set corresponding to the first image set, the text data set corresponding to the second image set, and the image element data set corresponding to the third image set as expected output data, to train the image recognition model.

[0017] In some embodiments, the image recognition model in the generation unit includes a container type recognition sub-model, a text recognition sub-model and an element recognition sub-model; the generation unit includes: a first generation module, configured to input the first image set into the container type recognition sub-model, and generate a container type data set corresponding to the first image set, wherein the container type recognition sub-model is used to characterize the container type determination of each image in the first image set; a second generation module, configured to input the second image set into the text recognition sub-model, and generate a text data set corresponding to the second image set, wherein the text recognition sub-model is used to characterize the text detection and text recognition of each image in the second image set; a third generation module, configured to input the third image set into the element recognition sub-model, and generate an image element data set corresponding to the third image set, wherein the element recognition sub-model is used to characterize the image element detection and recognition of each image in the third image set.

[0018] In some embodiments, the text recognition submodel in the second generation module includes a feature extraction submodel and a text sequence extraction submodel; the second generation module includes: a feature extraction submodule, configured to input the second image set into the feature extraction submodel to obtain each feature matrix corresponding to the second image set, wherein the feature extraction submodel is constructed based on a convolutional neural network; a text extraction submodule, configured to input each feature matrix into the text sequence extraction submodel to obtain a text sequence corresponding to each feature matrix, wherein the text sequence extraction submodel is constructed based on a recursive neural network; a determination submodule, configured to determine the text information corresponding to each text sequence based on each text sequence, and generate a text data set corresponding to each text information.

[0019] In some embodiments, the image recognition model in the generation unit and / or the container type recognition sub-model in the generation unit are constructed based on a deep residual network model.

[0020] In some embodiments, the device also includes: a correction unit, configured to correct the container type dataset, the text dataset, and the image element dataset to obtain a corrected container type dataset, the text dataset, and the image element dataset, wherein the correction is used to characterize the analysis results based on the image position, image sequence, and image repeatability of each image in each image set, and reorder the data in the container type dataset, the text dataset, and the image element dataset.

[0021] In some embodiments, the correction in the correction unit is performed based on a combination of image scaling, image grayscale, image enhancement, image noise reduction and image edge detection on each image in each image set.

[0022] In some embodiments, the device also includes: an identification unit, configured to perform content identification on each image set to obtain a first data set corresponding to the first image set, a second data set corresponding to the second image set, and a third data set corresponding to the third image set; a correction unit, configured to correct the data in the container type data set, the text data set, and the image element data set based on the comparison results of the first data set, the second data set, and the third data set with the container type data set, the text data set, and the image element data set to obtain corrected container type data sets, text data sets, and image element data sets.

[0023] In some embodiments, the device also includes: a display unit, configured to generate and display a template interface corresponding to the template data set based on the template data set; and / or an optimization unit, configured to optimize the design scheme of the page template based on the template data set.

[0024] According to the third aspect of the present application, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method described in any implementation manner in the first aspect.

[0025] According to a fourth aspect of the present application, the present application provides a non-transitory computer-readable storage medium storing computer instructions, characterized in that the computer instructions are used to enable a computer to execute a method as described in any implementation manner in the first aspect.

[0026] According to the technology of the present application, in response to receiving a page image, the page image is annotated, each image set corresponding to the annotated data is generated, each image set is input into the trained image recognition model, and a container type data set corresponding to the first image set, a text data set corresponding to the second image set, and an image element data set corresponding to the third image set are generated, wherein the image recognition model is used to characterize the container type determination of each image in the first image set, the text detection and text recognition of each image in the second image set, and the image element detection and recognition of each image in the third image set. Based on the template information of the page, the container type data set, the text data set, and the image element data set are converted to generate a template data set corresponding to the page image, and the template data set is uploaded. The page image is converted into template data by using image recognition technology and the template data is stored in the content distribution network by uploading data, avoiding the linear increase of the number of files as the template demand increases in the prior art, and solving the problem of poor reusability of JSON files and high maintenance cost in the process of page construction in the prior art. The accurate positioning of template data and efficient online template are realized, freeing the hands of maintenance personnel. The template data set is directly generated through image recognition technology, which saves system development resources and maintenance costs and improves the flexibility of template construction.

[0027] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present application.

[0029] Figure 1 is a schematic diagram of a first embodiment of a data processing method according to the present application;

[0030] Figure 2 It is a scene diagram that can implement the data processing method of the embodiment of the present application;

[0031] Figure 3 is a schematic diagram of a second embodiment of a data processing method according to the present application;

[0032] Figure 4 is a schematic structural diagram of an embodiment of a data processing device according to the present application;

[0033] Figure 5 It is a block diagram of an electronic device used to implement the data processing method of an embodiment of the present application. DETAILED DESCRIPTION

[0034] The following is a description of exemplary embodiments of the present application in conjunction with the accompanying drawings, including various details of the embodiments of the present application to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present application. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0035] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0036] Figure 1 A schematic diagram 100 of a first embodiment of a data processing method according to the present application is shown. The data processing method comprises the following steps:

[0037] Step 101 : in response to receiving a page image, annotate the page image and generate image sets corresponding to the annotated data.

[0038] In this embodiment, when the execution subject (such as a server or an intelligent terminal) receives a page image through a wired connection or a wireless connection, the page image can be annotated by a page crawler to generate each image set corresponding to the annotation data. Each image set may include: a first image set for identifying a container type, a second image set for identifying text information, and a third image set for detecting image elements. The page image can be generated based on a page template. The template can be generated based on the floor construction of a dynamic page, and the image sets can intersect, contain or be the same as each other. The template is the basic unit for building a dynamic page. The dynamic page floor display can be completed by configuring the template, and the same template can be reused multiple times on the page. It should be noted that the above-mentioned wireless connection method may include but is not limited to 3G, 4G, 5G connection, WiFi connection, Bluetooth connection, WiMAX connection, Zigbee connection, UWB (ultra wideband) connection, and other wireless connection methods currently known or to be developed in the future.

[0039] Step 102: Input each image set into the trained image recognition model to generate a container type data set corresponding to the first image set, a text data set corresponding to the second image set, and an image element data set corresponding to the third image set.

[0040] In this embodiment, the execution subject can input each image set into the trained image recognition model to generate a container type data set corresponding to the first image set, a text data set corresponding to the second image set, and an image element data set corresponding to the third image set. The image recognition model is used to characterize the container type determination of each image in the first image set, the text detection and text recognition of each image in the second image set, and the image element detection and recognition of each image in the third image set. The image recognition model is trained by the historical relevant data of each image set.

[0041] In some optional implementations, the image recognition model is trained in the following manner: obtaining a training sample set, wherein the training samples in the training sample set include a first image set for identifying container types, a second image set for identifying text information, a third image set for detecting image elements, a container type data set corresponding to the first image set, a text data set corresponding to the second image set, and an image element data set corresponding to the third image set; using a deep learning method, taking the first image set, the second image set, and the third image set included in the training samples in the training sample set as input data, and taking the container type data set corresponding to the first image set, the text data set corresponding to the second image set, and the image element data set corresponding to the third image set as expected output data, to train the image recognition model. Using deep learning technology for model training makes model prediction more accurate and comprehensive.

[0042] Step 103: based on the template information of the page, convert the container type dataset, the text dataset and the image element dataset to generate a template dataset corresponding to the page image, and upload the template dataset.

[0043] In this embodiment, the execution subject can convert the container type data set, text data set and image element data set using the data conversion method based on the template information of the page, generate a template data set corresponding to the page image, and upload the template data set. The conversion converts the container type data set, text data set and image element data set based on a specific language structure, for example, converting the container type data set, text data set and image element data set into a domain-specific language (DSL) to achieve data unification, and the unified data is uploaded to the content delivery network (CDN) for content storage, so as to be updated and maintained through a visual construction interface.

[0044] It should be noted that technicians can set the model structure of the above-mentioned image recognition model according to actual needs, and the embodiments of the present disclosure are not limited to this.

[0045] Continue to see Figure 2 , the data processing method 200 of this embodiment runs in the service platform 201. When the service platform 201 receives the page image, it annotates the page image and generates each image set 202 corresponding to the annotated data. Then the service platform 201 inputs each image set into the trained image recognition model to generate a container type data set corresponding to the first image set, a text data set corresponding to the second image set, and an image element data set 203 corresponding to the third image set. Then the service platform 201 converts the container type data set, the text data set, and the image element data set based on the template information of the page, generates a template data set corresponding to the page image, and uploads the template data set 204. Among them, each image set includes: a first image set for identifying the container type, a second image set for identifying the text information, and a third image set for detecting the image element. The page image is generated based on the page template. The image recognition model is used to characterize the container type determination of each image in the first image set, the text detection and text recognition of each image in the second image set, and the image element detection and recognition of each image in the third image set.

[0046] The data processing method provided by the above embodiment of the present application adopts a method of annotating the page image in response to receiving the page image, generating each image set corresponding to the annotated data, inputting each image set into the trained image recognition model, generating a container type data set corresponding to the first image set, a text data set corresponding to the second image set, and an image element data set corresponding to the third image set, wherein the image recognition model is used to characterize the container type determination of each image in the first image set, the text detection and text recognition of each image in the second image set, and the image element detection and recognition of each image in the third image set, based on the template information of the page, the container type data set, the text data set and the image element data set are converted to generate a template data set corresponding to the page image, and the template data set is uploaded, and the page image is converted into template data by using image recognition technology and the template data is stored in the content distribution network by uploading data, thereby avoiding the situation in the prior art that the number of files increases linearly with the increase in template demand, and solving the problem of poor reusability of JSON files and excessively high maintenance costs during page construction in the prior art. The accurate positioning of template data and efficient online templates are achieved, freeing the hands of maintenance personnel. The template data set is directly generated through image recognition technology, which saves system development resources and maintenance costs and improves the flexibility of template construction.

[0047] Further references Figure 3 , which shows a schematic diagram 300 of a second embodiment of a data processing method. The process of the method comprises the following steps:

[0048] Step 301 : in response to receiving a page image, annotate the page image and generate image sets corresponding to the annotated data.

[0049] In some optional implementations of this embodiment, the page image is annotated to generate various image sets corresponding to the annotated data, including: annotating the page image to obtain the annotated data corresponding to the page image; inputting the annotated data into a position determination model to generate the position information of each block corresponding to the annotated data, wherein the position determination model is trained by historical related data of the annotated data; based on the position information of each block, determining each image set corresponding to the annotated data. The position determination model can use the content analysis algorithm readability to calculate the most likely block position information according to the different weights of the annotated data. This method can achieve a more accurate effect in locating the effective block.

[0050] Step 302: input the first image set into the container type recognition sub-model to generate a container type data set corresponding to the first image set; input the second image set into the text recognition sub-model to generate a text data set corresponding to the second image set; input the third image set into the element recognition sub-model to generate an image element data set corresponding to the third image set.

[0051] In this embodiment, the image recognition model may include a container type recognition submodel, a text recognition submodel, and an element recognition submodel. The execution entity may input the first image set into the container type recognition submodel to generate a container type data set corresponding to the first image set, input the second image set into the text recognition submodel to generate a text data set corresponding to the second image set, and input the third image set into the element recognition submodel to generate an image element data set corresponding to the third image set. The container type recognition submodel is used to characterize the container type determination of each image in the first image set, the text recognition submodel is used to characterize the text detection and text recognition of each image in the second image set, and the element recognition submodel is used to characterize the image element detection and recognition of each image in the third image set. The image recognition model and the container type recognition submodel are constructed based on a deep residual network model. The deep residual network (ResNet) is used to solve the problem that the performance of the neural network degrades significantly with the increase of depth.

[0052] In some optional implementations of this embodiment, the text recognition submodel includes a feature extraction submodel and a text sequence extraction submodel; the second image set is input into the text recognition submodel to generate a text data set corresponding to the second image set, including: inputting the second image set into the feature extraction submodel to obtain each feature matrix corresponding to the second image set, wherein the feature extraction submodel is constructed based on a convolutional neural network; inputting each feature matrix into the text sequence extraction submodel to obtain a text sequence corresponding to each feature matrix, wherein the text sequence extraction submodel is constructed based on a recursive neural network; based on each text sequence, determining the text information corresponding to each text sequence, and generating a text data set corresponding to each text information. Text recognition uses a convolutional neural network CNN algorithm for feature extraction, and overcomes image rotation and local subtle changes through pooling operations, and then predicts label division according to a recursive neural network RNN, models changes in time series, and transmits serialized messages, and finally uses a sequence loss function (Connectionist Temporal Classification, CTC loss) as an objective function for optimization. CTC loss is a loss function in sequence labeling problems, which is mainly used to deal with the input and output label alignment problem in sequence labeling problems.

[0053] Step 303: Correct the container type dataset, the text dataset, and the image element dataset to obtain corrected container type dataset, text dataset, and image element dataset.

[0054] In this implementation, the execution subject can correct the container type data set, text data set and image element data set to obtain the corrected container type data set, text data set and image element data set. Correction is used to characterize the analysis results based on the image position, image sequence and image repeatability of each image in each image set, and reorder the data in the container type data set, text data set and image element data set. By detecting and correcting the located and recognized data, the accuracy of the data is improved.

[0055] To further illustrate, the execution entity may measure the image elements in each image set based on a morphological transformation method to obtain contour information of the element frame; correct the contour information of the element frame using a position correction method; align the contour information of the corrected element frame, wherein the alignment represents the alignment of the horizontal coordinates and / or vertical coordinates of the element frame; and reorder the aligned element frames to obtain an ordered container type data set, a text data set, and an image element data set.

[0056] In some optional implementations of this embodiment, the correction is performed based on a combination of image scaling, image graying, image enhancement, image noise reduction, and image edge detection for each image in each image set. It should be noted that the above-mentioned various image processing methods are well-known technologies that are currently widely studied and applied, and will not be described in detail here. The combined use of correction formulas and parameter settings are obtained by developers through practice, which improves system efficiency and accuracy.

[0057] In some optional implementations of this embodiment, before correcting the container type data set, text data set and image element data set to obtain the corrected container type data set, text data set and image element data set, it also includes: performing content recognition on each image set by a content recognition method to obtain a first data set corresponding to the first image set, a second data set corresponding to the second image set and a third data set corresponding to the third image set; and correcting the data in the container type data set, text data set and image element data set according to the comparison results of the first data set, the second data set and the third data set with the container type data set, text data set and image element data set to obtain the corrected container type data set, text data set and image element data set. By obtaining the traditional image processing results, the data is corrected multiple times by combining the depth detection results with the traditional image processing results, thereby improving the data accuracy.

[0058] Step 304: based on the template information of the page, convert the container type dataset, the text dataset and the image element dataset to generate a template dataset corresponding to the page image, and upload the template dataset.

[0059] In some optional implementations of this embodiment, the method further includes: based on the template data set, generating and displaying a template interface corresponding to the template data set, thereby realizing the construction of a fast and flexible cross-front-end application of active templates.

[0060] In some optional implementations of this embodiment, the method further includes: optimizing the design of the page template based on the template data set, realizing the online configuration capability of the template production line that mixes and matches the template style and template data, realizing the capability of providing a better template solution for the existing online page, and further improving the product conversion rate.

[0061] In this embodiment, the specific operations of steps 301 and 304 are the same as Figure 1 The operations of steps 101 and 103 in the illustrated embodiment are substantially the same and will not be described in detail herein.

[0062] from Figure 3 It can be seen that Figure 1Compared with the corresponding embodiment, the schematic diagram 300 of the data processing method in this embodiment adopts inputting the first image set into the container type recognition sub-model to generate a container type data set corresponding to the first image set, inputting the second image set into the text recognition sub-model to generate a text data set corresponding to the second image set, inputting the third image set into the element recognition sub-model to generate an image element data set corresponding to the third image set, correcting the container type data set, the text data set and the image element data set to obtain the corrected container type data set, the text data set and the image element data set, and obtaining the container type data set, the text data set and the image element data set based on different models respectively, so that data processing is more accurate and targeted, and the residual network design model is adopted to solve the problem of model gradient disappearance, thereby improving the accuracy of model training.

[0063] Further references Figure 4 , as a response to the above Figures 1 to 3 In order to realize the method shown in the figure, the present application provides an embodiment of a data processing device, and the device embodiment is Figure 1 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0064] like Figure 4 As shown, the data processing device 400 of this embodiment includes: a labeling unit 401, a generating unit 402 and a converting unit 403, wherein the labeling unit is configured to label the page image in response to receiving the page image, and generate each image set corresponding to the labeling data, wherein each image set includes: a first image set for identifying the container type, a second image set for identifying text information and a third image set for detecting image elements, and the page image is generated based on the page template; the generating unit is configured to input each image set into the trained image recognition model, generate a container type data set corresponding to the first image set, a second image set for identifying text information and a third image set for detecting image elements, and the page image is generated based on the page template; the generating unit is configured to input each image set into the trained image recognition model, generate a container type data set corresponding to the first image set, a second image set for identifying text information and a third image set for detecting image elements A text dataset corresponding to the first image set and an image element dataset corresponding to the third image set, wherein the image recognition model is used to characterize the container type determination of each image in the first image set, the text detection and text recognition of each image in the second image set, and the image element detection and recognition of each image in the third image set; a conversion unit is configured to convert the container type dataset, the text dataset, and the image element dataset based on the template information of the page, generate a template dataset corresponding to the page image, and upload the template dataset, wherein the conversion converts the container type dataset, the text dataset, and the image element dataset based on a specific language structure.

[0065] In this embodiment, the specific processing of the labeling unit 401, the generating unit 402 and the converting unit 403 of the data processing device 400 and the technical effects thereof can be referred to in Figure 1The relevant descriptions of steps 101 to 103 in the corresponding embodiment are not repeated here.

[0066] In some optional implementations of the present embodiment, the labeling unit includes: a labeling module, configured to label the page image and obtain labeling data corresponding to the page image; a position generation module, configured to input the labeling data into a position determination model, and generate position information of each block corresponding to the labeling data, wherein the position determination model is trained by historical related data of the labeling data; and a determination module, configured to determine each image set corresponding to the labeling data based on the position information of each block.

[0067] In some optional implementations of the present embodiment, the image recognition model in the generation unit is trained using the following modules: an acquisition module, configured to acquire a training sample set, wherein the training samples in the training sample set include a first image set for identifying a container type, a second image set for identifying text information, a third image set for detecting image elements, a container type data set corresponding to the first image set, a text data set corresponding to the second image set, and an image element data set corresponding to the third image set; a training module, configured to use a deep learning method, using the first image set, the second image set, and the third image set included in the training samples in the training sample set as input data, and using the container type data set corresponding to the first image set, the text data set corresponding to the second image set, and the image element data set corresponding to the third image set as expected output data, to train the image recognition model.

[0068] In some optional implementations of the present embodiment, the image recognition model in the generation unit includes a container type recognition sub-model, a text recognition sub-model and an element recognition sub-model; the generation unit includes: a first generation module, configured to input the first image set into the container type recognition sub-model, and generate a container type data set corresponding to the first image set, wherein the container type recognition sub-model is used to characterize the container type determination for each image in the first image set; a second generation module, configured to input the second image set into the text recognition sub-model, and generate a text data set corresponding to the second image set, wherein the text recognition sub-model is used to characterize the text detection and text recognition for each image in the second image set; a third generation module, configured to input the third image set into the element recognition sub-model, and generate an image element data set corresponding to the third image set, wherein the element recognition sub-model is used to characterize the image element detection and recognition for each image in the third image set.

[0069] In some optional implementations of the present embodiment, the text recognition submodel in the second generation module includes a feature extraction submodel and a text sequence extraction submodel; the second generation module includes: a feature extraction submodule, configured to input the second image set into the feature extraction submodel, and obtain each feature matrix corresponding to the second image set, wherein the feature extraction submodel is constructed based on a convolutional neural network; a text extraction submodule, configured to input each feature matrix into the text sequence extraction submodel, and obtain a text sequence corresponding to each feature matrix, wherein the text sequence extraction submodel is constructed based on a recursive neural network; a determination submodule, configured to determine the text information corresponding to each text sequence based on each text sequence, and generate a text data set corresponding to each text information.

[0070] In some optional implementations of this embodiment, the image recognition model in the generation unit and / or the container type recognition sub-model in the generation unit are constructed based on a deep residual network model.

[0071] In some optional implementations of the present embodiment, the device further includes: a correction unit configured to correct the container type dataset, the text dataset, and the image element dataset to obtain corrected container type dataset, text dataset, and image element dataset, wherein the correction is used to characterize the analysis results based on the image position, image sequence, and image repeatability of each image in each image set, and reorder the data in the container type dataset, the text dataset, and the image element dataset.

[0072] In some optional implementations of this embodiment, the correction in the correction unit is completed based on a combination of image scaling, image grayscale, image enhancement, image denoising and image edge detection for each image in each image set.

[0073] In some optional implementations of the present embodiment, the device also includes: an identification unit, configured to perform content identification on each image set to obtain a first data set corresponding to the first image set, a second data set corresponding to the second image set, and a third data set corresponding to the third image set; a correction unit, configured to correct the data in the container type data set, the text data set, and the image element data set based on the comparison results of the first data set, the second data set, and the third data set with the container type data set, the text data set, and the image element data set to obtain corrected container type data sets, text data sets, and image element data sets.

[0074] In some optional implementations of this embodiment, the device also includes: a display unit, configured to generate and display a template interface corresponding to the template data set based on the template data set; and / or an optimization unit, configured to optimize the design scheme of the page template based on the template data set.

[0075] According to an embodiment of the present application, the present application also provides an electronic device and a readable storage medium.

[0076] like Figure 5 , is a block diagram of an electronic device according to a data processing method according to an embodiment of the present application. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or required herein.

[0077] like Figure 5 As shown, the electronic device includes: one or more processors 501, a memory 502, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. The various components are interconnected using different buses and can be installed on a common mainboard or installed in other ways as needed. The processor can process instructions executed in the electronic device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as a display device coupled to the interface). In other embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple electronic devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 5 A processor 501 is taken as an example.

[0078] The memory 502 is a non-transient computer-readable storage medium provided in the present application. The memory stores instructions executable by at least one processor to enable at least one processor to perform the data processing method provided in the present application. The non-transient computer-readable storage medium of the present application stores computer instructions, which are used to enable a computer to perform the data processing method provided in the present application.

[0079] The memory 502 is a non-transient computer-readable storage medium that can be used to store non-transient software programs, non-transient computer executable programs and modules, such as program instructions / modules corresponding to the data processing method in the embodiment of the present application (for example, the attached Figure 4The processor 501 executes various functional applications and data processing of the server by running the non-transient software programs, instructions and modules stored in the memory 502, that is, the data processing method in the above method embodiment is implemented.

[0080] The memory 502 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required by at least one function; the data storage area may store data created according to the use of the data processing electronic device, etc. In addition, the memory 502 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory 502 may optionally include a memory remotely arranged relative to the processor 501, and these remote memories may be connected to the data processing electronic device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0081] The electronic device of the data processing method may further include: an input device 503 and an output device 504. The processor 501, the memory 502, the input device 503 and the output device 504 may be connected via a bus or other means. Figure 5 The example of connecting through bus is taken in the following.

[0082] The input device 503 can receive input digital or character information, and generate key signal input related to user settings and function control of the data processing electronic device, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, an indicator bar, one or more mouse buttons, a trackball, a joystick and other input devices. The output device 504 may include a display device, an auxiliary lighting device (e.g., an LED) and a tactile feedback device (e.g., a vibration motor), etc. The display device may include, but is not limited to, a liquid crystal display (LCD), a light emitting diode (LED) display and a plasma display. In some embodiments, the display device may be a touch screen.

[0083] Various implementations of the systems and techniques described herein can be realized in digital electronic circuit systems, integrated circuit systems, dedicated ASICs (application specific integrated circuits), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0084] These computer programs (also referred to as programs, software, software applications, or code) include machine instructions for programmable processors and can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, device, and / or means (e.g., disk, optical disk, memory, programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.

[0085] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0086] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0087] A computer system may include clients and servers. Clients and servers are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship to each other.

[0088] According to the technical solution of the embodiment of the present application, in response to receiving a page image, the page image is annotated, each image set corresponding to the annotated data is generated, each image set is input into the trained image recognition model, and a container type data set corresponding to the first image set, a text data set corresponding to the second image set, and an image element data set corresponding to the third image set are generated, wherein the image recognition model is used to characterize the container type determination of each image in the first image set, the text detection and text recognition of each image in the second image set, and the image element detection and recognition of each image in the third image set. Based on the template information of the page, the container type data set, the text data set, and the image element data set are converted to generate a template data set corresponding to the page image, and the template data set is uploaded. The page image is converted into template data by using image recognition technology and the template data is stored in the content distribution network by uploading data, thereby avoiding the linear increase in the number of files as the template demand increases in the prior art, and solving the problem of poor reusability of JSON files and high maintenance costs in the process of page construction in the prior art. The accurate positioning of template data and efficient online templates are achieved, freeing the hands of maintenance personnel. The template data set is directly generated through image recognition technology, which saves system development resources and maintenance costs and improves the flexibility of template construction.

[0089] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this application can be executed in parallel, sequentially or in different orders, as long as the expected results of the technical solution disclosed in this application can be achieved, and this document is not limited here.

[0090] The above specific implementations do not constitute a limitation on the protection scope of this application. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of this application should be included in the protection scope of this application.

Claims

1. A data processing method, the method comprising: In response to receiving a page image, annotating the page image and generating respective image sets corresponding to the annotation data, wherein the respective image sets include: a first image set for identifying a container type, a second image set for identifying text information, and a third image set for detecting image elements, and the page image is generated based on a page template; Input each image set into the trained image recognition model to generate a container type data set corresponding to the first image set, a text data set corresponding to the second image set, and an image element data set corresponding to the third image set, wherein the image recognition model is used to characterize the container type determination of each image in the first image set, the text detection and text recognition of each image in the second image set, and the image element detection and recognition of each image in the third image set; Based on the template information of the page, the container type dataset, the text dataset and the image element dataset are converted to generate a template dataset corresponding to the page image, and the template dataset is uploaded, wherein the conversion converts the container type dataset, the text dataset and the image element dataset based on a specific language structure.

2. The method according to claim 1, wherein: The step of annotating the page image to generate image sets corresponding to the annotated data includes: Annotating the page image to obtain annotation data corresponding to the page image; Inputting the annotated data into a position determination model to generate position information of each block corresponding to the annotated data, wherein the position determination model is trained by historical related data of the annotated data; Based on the position information of each block, each image set corresponding to the annotation data is determined.

3. The method according to claim 1, wherein: The image recognition model is trained in the following way: Acquire a training sample set, wherein the training samples in the training sample set include a first image set for identifying container types, a second image set for identifying text information, a third image set for detecting image elements, a container type data set corresponding to the first image set, a text data set corresponding to the second image set, and an image element data set corresponding to the third image set; Using a deep learning method, the first image set, the second image set and the third image set included in the training samples in the training sample set are taken as input data, and the container type data set corresponding to the first image set, the text data set corresponding to the second image set and the image element data set corresponding to the third image set are taken as expected output data to train an image recognition model.

4. The method according to claim 1, wherein: The image recognition model includes a container type recognition sub-model, a text recognition sub-model and an element recognition sub-model; the step of inputting each image set into the trained image recognition model to generate a container type data set corresponding to the first image set, a text data set corresponding to the second image set and an image element data set corresponding to the third image set includes: Inputting the first image set into the container type recognition sub-model to generate a container type data set corresponding to the first image set, wherein the container type recognition sub-model is used to characterize the container type determination of each image in the first image set; Inputting the second image set into the text recognition sub-model to generate a text data set corresponding to the second image set, wherein the text recognition sub-model is used to characterize text detection and text recognition for each image in the second image set; The third image set is input into the element recognition sub-model to generate an image element data set corresponding to the third image set, wherein the element recognition sub-model is used to characterize image element detection and recognition of each image in the third image set.

5. The method according to claim 4, wherein: The text recognition sub-model includes a feature extraction sub-model and a text sequence extraction sub-model; The step of inputting the second image set into the text recognition sub-model to generate a text data set corresponding to the second image set includes: Inputting the second image set into the feature extraction sub-model to obtain each feature matrix corresponding to the second image set, wherein the feature extraction sub-model is constructed based on a convolutional neural network; Inputting each feature matrix into the text sequence extraction sub-model to obtain a text sequence corresponding to each feature matrix, wherein the text sequence extraction sub-model is constructed based on a recursive neural network; Based on each of the character sequences, text information corresponding to each of the character sequences is determined, and a text data set corresponding to each of the text information is generated.

6. The method according to claim 4, wherein: The image recognition model and / or the container type recognition sub-model are constructed based on a deep residual network model.

7. The method according to claim 1, wherein: Before converting the container type dataset, the text dataset and the image element dataset based on the template information of the page to generate a template dataset corresponding to the page image, the method further includes: The container type dataset, the text dataset and the image element dataset are corrected to obtain the corrected container type dataset, the text dataset and the image element dataset, wherein the correction is used to characterize the analysis results based on the image position, image sequence and image repeatability of each image in each image set, and reorder the data in the container type dataset, the text dataset and the image element dataset.

8. The method according to claim 7, wherein: The correction is completed based on a combination of image scaling, image graying, image enhancement, image noise reduction and image edge detection for each image in each image set.

9. The method according to claim 7, wherein: Before correcting the container type dataset, the text dataset and the image element dataset to obtain the corrected container type dataset, the text dataset and the image element dataset, the method further includes: Performing content recognition on each image set to obtain a first data set corresponding to the first image set, a second data set corresponding to the second image set, and a third data set corresponding to the third image set; According to the comparison results of the first data set, the second data set and the third data set with the container type data set, the text data set and the image element data set, the data in the container type data set, the text data set and the image element data set are corrected to obtain the corrected container type data set, the text data set and the image element data set.

10. The method according to claim 1, further comprising: Based on the template data set, a template interface corresponding to the template data set is generated and displayed; and / or, Based on the template data set, the design scheme of the page template is optimized.

11. A data processing device, comprising: a marking unit configured to mark the page image in response to receiving the page image, and generate various image sets corresponding to the marking data, wherein the various image sets include: a first image set for identifying a container type, a second image set for identifying text information, and a third image set for detecting image elements, and the page image is generated based on a page template; a generating unit configured to input each image set into a trained image recognition model, and generate a container type data set corresponding to the first image set, a text data set corresponding to the second image set, and an image element data set corresponding to the third image set, wherein the image recognition model is used to characterize the container type determination for each image in the first image set, the text detection and text recognition for each image in the second image set, and the image element detection and recognition for each image in the third image set; A conversion unit is configured to convert the container type dataset, the text dataset and the image element dataset based on the template information of the page, generate a template dataset corresponding to the page image, and upload the template dataset, wherein the conversion converts the container type dataset, the text dataset and the image element dataset based on a specific language structure.

12. The device according to claim 11, wherein: The marking unit comprises: a marking module, configured to mark the page image to obtain marking data corresponding to the page image; a position generation module, configured to input the annotated data into a position determination model to generate position information of each block corresponding to the annotated data, wherein the position determination model is trained by historical related data of the annotated data; The determination module is configured to determine each image set corresponding to the annotation data based on the position information of each block.

13. The device according to claim 11, wherein: The image recognition model in the generation unit is trained using the following modules: an acquisition module, configured to acquire a training sample set, wherein the training samples in the training sample set include a first image set for identifying container types, a second image set for identifying text information, a third image set for detecting image elements, a container type data set corresponding to the first image set, a text data set corresponding to the second image set, and an image element data set corresponding to the third image set; The training module is configured to use a deep learning method to take the first image set, the second image set and the third image set included in the training samples in the training sample set as input data, and take the container type data set corresponding to the first image set, the text data set corresponding to the second image set and the image element data set corresponding to the third image set as expected output data, to train an image recognition model.

14. The device according to claim 11, wherein: The image recognition model in the generation unit includes a container type recognition sub-model, a text recognition sub-model and an element recognition sub-model; The generating unit comprises: A first generating module is configured to input the first image set into the container type recognition sub-model to generate a container type data set corresponding to the first image set, wherein the container type recognition sub-model is used to characterize the container type determination of each image in the first image set; A second generation module is configured to input the second image set into the text recognition sub-model to generate a text data set corresponding to the second image set, wherein the text recognition sub-model is used to characterize text detection and text recognition for each image in the second image set; The third generation module is configured to input the third image set into the element recognition sub-model to generate an image element data set corresponding to the third image set, wherein the element recognition sub-model is used to characterize image element detection and recognition of each image in the third image set.

15. The device according to claim 14, wherein: The text recognition sub-model in the second generation module includes a feature extraction sub-model and a text sequence extraction sub-model; The second generating module comprises: a feature extraction submodule, configured to input the second image set into the feature extraction submodel to obtain each feature matrix corresponding to the second image set, wherein the feature extraction submodel is constructed based on a convolutional neural network; A text extraction submodule is configured to input each feature matrix into the text sequence extraction submodel to obtain a text sequence corresponding to each feature matrix, wherein the text sequence extraction submodel is constructed based on a recursive neural network; The determination submodule is configured to determine the text information corresponding to each of the text sequences based on each of the text sequences, and generate a text data set corresponding to each of the text information.

16. The device according to claim 14, wherein: The image recognition model in the generation unit and / or the container type recognition sub-model in the generation unit are constructed based on a deep residual network model.

17. The device according to claim 11, further comprising: A correction unit is configured to correct the container type dataset, the text dataset and the image element dataset to obtain the corrected container type dataset, the text dataset and the image element dataset, wherein the correction is used to characterize the analysis results based on the image position, image sequence and image repeatability of each image in each image set, and reorder the data in the container type dataset, the text dataset and the image element dataset.

18. The device according to claim 17, wherein: The correction in the correction unit is completed based on a combination of image scaling, image graying, image enhancement, image noise reduction and image edge detection for each image in each image set.

19. The device according to claim 17, further comprising: an identification unit configured to perform content identification on each image set to obtain a first data set corresponding to the first image set, a second data set corresponding to the second image set, and a third data set corresponding to the third image set; The correction unit is configured to correct the data in the container type data set, the text data set and the image element data set according to the comparison result of the first data set, the second data set and the third data set with the container type data set, the text data set and the image element data set to obtain the corrected container type data set, the text data set and the image element data set.

20. The device according to claim 11, further comprising: A display unit, configured to generate and display a template interface corresponding to the template data set based on the template data set; and / or, The optimization unit is configured to optimize the design of the page template based on the template data set.

21. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 10.

22. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 10.

Images