Layout analysis model determination method and apparatus, electronic device, and storage medium

Abstract

Embodiments of the present application disclose a layout analysis model determination method and device, electronic equipment and a medium. Based on a pre-trained layout analysis model, at least one pseudo label of an unmarked layout sample is determined to obtain a candidate pseudo label sample with the pseudo label; for each kind of pseudo label, based on an initial feature extraction model of the kind of pseudo label, a hypersphere center corresponding to the kind of pseudo label is determined, and according to the hypersphere center and a marked layout sample corresponding to the kind of pseudo label, a target hypersphere radius of the pseudo label is determined; according to the target hypersphere radius of the kind of pseudo label, a target pseudo label sample is selected from the candidate pseudo label sample corresponding to the kind of pseudo label; and according to the target pseudo label sample of each kind of pseudo label, the pre-trained layout analysis model is trained to obtain a target layout analysis model. The embodiments of the present application reduce the dependence on label samples and improve the accuracy of layout analysis.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the image processing technology, and in particular, to a layout analysis model determination method and device, electronic equipment and storage medium. BACKGROUND

[0002] Layout analysis of a document image can realize positioning and recognition of different regions of document content and plays an important role in accurately obtaining important region information of a document. Through a traditional layout analysis method, a technician needs to manually design an extraction method of a feature, needs to have a large amount of prior knowledge, consumes a large amount of manpower and has a high labor cost.

[0003] With the development of deep learning methods, more and more deep learning methods are also applied to layout analysis. However, deep learning needs a large number of labeled samples for supervised training and has a large dependence on sample labels. When the number of labeled samples is small, the accuracy is low. SUMMARY

[0004] The present application provides a layout analysis model determination method and device, electronic equipment and storage medium to reduce the dependence on labeled samples and improve the accuracy of layout analysis.

[0005] In a first aspect, the embodiments of the present application provide a layout analysis model determination method, which comprises the following steps:

[0006] Based on a pre-trained layout analysis model, at least one layout pseudo label of an unlabeled layout sample is determined to obtain a candidate pseudo label sample with a layout pseudo label; the pre-trained layout analysis model is a deep learning model pre-trained by using labeled layout samples;

[0007] For each kind of layout pseudo label, based on an initial feature extraction model of the kind of layout pseudo label, a hypersphere center corresponding to the kind of layout pseudo label is determined;

[0008] For each kind of layout pseudo label, based on an initial feature extraction model of the kind of layout pseudo label, a target hypersphere radius of the kind of layout pseudo label is determined according to a labeled layout sample corresponding to the kind of layout pseudo label and the hypersphere center;

[0009] According to the target hypersphere radius of the kind of layout pseudo label, a target pseudo label sample is selected from the candidate pseudo label sample corresponding to the kind of layout pseudo label;

[0010] According to the target pseudo label sample of each kind of layout pseudo label, the pre-trained layout analysis model is trained to obtain a target layout analysis model.

[0011] In a second aspect, the embodiments of the present application further provide a layout analysis model determination apparatus, the layout analysis model determination apparatus comprising:

[0012] a layout pseudo-label marking module configured to determine at least one layout pseudo-label of an unmarked layout sample based on a pre-trained layout analysis model, to obtain a candidate pseudo-label sample with the layout pseudo-label, wherein the pre-trained layout analysis model is a deep learning model pre-trained using marked layout samples;

[0013] a hypersphere center determination module configured to determine, for each type of layout pseudo-label, a hypersphere center corresponding to the type of layout pseudo-label based on an initial feature extraction model of the type of layout pseudo-label;

[0014] a target hypersphere radius determination module configured to determine, for each type of layout pseudo-label, a target hypersphere radius of the type of layout pseudo-label based on the initial feature extraction model of the type of layout pseudo-label, according to a marked layout sample and the hypersphere center corresponding to the type of layout pseudo-label;

[0015] a target pseudo-label sample screening module configured to screen a target pseudo-label sample from the candidate pseudo-label sample corresponding to the type of layout pseudo-label according to the target hypersphere radius of the type of layout pseudo-label;

[0016] a target layout analysis model determination module configured to train the pre-trained layout analysis model according to the target pseudo-label sample of each type of layout pseudo-label, to obtain a target layout analysis model.

[0017] In a third aspect, the embodiments of the present application further provide an electronic device, the electronic device comprising:

[0018] one or more processors;

[0019] a storage apparatus configured to store one or more programs;

[0020] when the one or more programs are executed by the one or more processors, the one or more processors are caused to implement any one of the layout analysis model determination methods provided by the embodiments of the present application.

[0021] In a fourth aspect, the embodiments of the present application further provide a storage medium comprising computer executable instructions, when executed by a computer processor, for performing any one of the layout analysis model determination methods provided by the embodiments of the present application.

[0022] The application determines at least one layout pseudo label of an unmarked layout sample by a pre-trained layout analysis model to obtain a candidate pseudo label sample with a layout pseudo label; the pre-trained layout analysis model is a deep learning model pre-trained by using a marked layout sample; the pre-trained layout analysis model can be obtained by pre-training the deep learning model by the marked layout sample, and can analyze the unmarked layout sample to obtain the layout pseudo label, so as to increase the number of marked samples; for each kind of layout pseudo label, an initial feature extraction model based on the kind of layout pseudo label is used to determine the hyper-spherical center corresponding to the kind of layout pseudo label; for each kind of layout pseudo label, an initial feature extraction model based on the kind of layout pseudo label is used to determine the target hyper-spherical radius of the kind of layout pseudo label according to the marked layout sample corresponding to the kind of layout pseudo label and the hyper-spherical center; the hyper-spherical center is obtained by the marked layout sample, and then the hyper-spherical radius is obtained; the accuracy of the layout pseudo label can be judged according to the distribution of the marked layout sample; the target pseudo label sample is selected from the candidate pseudo label sample corresponding to the kind of layout pseudo label according to the target hyper-spherical radius of the kind of layout pseudo label, so as to increase the number of training samples of the pre-trained layout analysis model; the pre-trained layout analysis model is trained according to the target pseudo label sample of each kind of layout pseudo label to obtain a target layout analysis model; the data amount of training can be improved by the obtained target pseudo label sample, and the accuracy of layout analysis can be improved. Therefore, by the technical scheme of the application, the problem that deep learning needs a large number of labeled samples for supervised training and has a large dependence on sample labels, and the accuracy is low when the number of labeled samples is small, is solved, and the effect of reducing the dependence of the deep learning model on the labeled samples and improving the accuracy of layout analysis is achieved. BRIEF DESCRIPTION OF DRAWINGS

[0023] Figure 1 is a flowchart of a layout analysis model determination method in the first embodiment of the application;

[0024] Figure 2 is a flowchart of a layout analysis model determination method in the second embodiment of the application;

[0025] Figure 3 is a structural schematic diagram of a layout analysis model determination device in the third embodiment of the application;

[0026] Figure 4 is a structural schematic diagram of an electronic device in the fourth embodiment of the application. DETAILED DESCRIPTION

[0027] In the following, the technical solutions in the embodiments of the present application will be described clearly and completely in conjunction with the drawings in the embodiments of the present application, so that those skilled in the art can better understand the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work should fall within the scope of the present application.

[0028] It should be noted that the terms "first" and "second" and the like in the specification and claims of the present application and the above-described drawings are used to distinguish similar objects, and do not necessarily have to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units does not have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0029] Embodiment one

[0030] Figure 1 A flowchart of a method for determining a layout analysis model is provided for the first embodiment of the present application. The present embodiment can be applicable to training a deep learning network with fewer labeled samples, so that the trained deep learning network can accurately perform layout analysis on layout images. The method can be performed by a determination device of the layout analysis model. The device can be implemented by software and / or hardware, and can be specifically configured in an electronic device, such as a computer.

[0031] Referring to Figure 1 The method for determining a layout analysis model specifically includes the following steps:

[0032] S110, determining at least one layout pseudo-label of an unlabeled layout sample based on a pre-trained layout analysis model, to obtain a candidate pseudo-label sample with a layout pseudo-label; the pre-trained layout analysis model is a deep learning model pre-trained using labeled layout samples.

[0033] The pre-trained layout analysis model can be a deep learning network trained by a small amount of labeled layout samples, and used for labeling unlabeled layout samples. For example, the pre-trained layout analysis model can be a Faster R-CNN (a professional term, a kind of deep learning model). The unlabeled layout sample is a layout sample without label marking. In practice, labeling a layout requires a large amount of manpower, and therefore the number of labeled layout samples is insufficient, which cannot sufficiently train the layout analysis model. The pre-trained layout analysis model is obtained by training the layout analysis model built by the labeled layout samples.

[0034] The layout pseudo label can be a layout label obtained by the pre-trained layout analysis model, and used to obtain a candidate label sample. For example, the layout pseudo label can include a title, a text, a table and a formula, etc., which are not limited in the present application. The unlabeled layout sample is input into the pre-trained layout analysis model, the pre-trained layout analysis model detects the input unlabeled layout sample, and adds a layout pseudo label to the information of different regions in the layout. For example, the layout pseudo label can be a frame with different colors, different shapes or different thicknesses. For example, if the layout pseudo label is a frame with different colors, the layout pseudo label can be determined as follows: a yellow frame is added to the detected title region, a blue frame is added to the detected table region, and a red frame is added to the detected formula region.

[0035] The candidate pseudo label sample can be sample data obtained by cropping the layout image according to the layout pseudo label. Specifically, the unlabeled layout sample is cropped according to the layout pseudo label to obtain the candidate pseudo label sample. In order to increase the number of samples, the candidate pseudo label sample can be data enhanced. Specifically, the data enhancement of the candidate pseudo label sample can be rotation, translation and scaling operations, etc.

[0036] In S120, for each kind of layout pseudo label, an initial feature extraction model of the kind of layout pseudo label is used to determine a hyper-sphere center corresponding to the kind of layout pseudo label.

[0037] The initial feature extraction model can be a deep learning model trained by the labeled layout sample, and used to determine the hyper-sphere center of each kind of layout pseudo label. Each kind of layout pseudo label corresponds to an initial feature extraction model. For example, the initial feature extraction model can be a ResNet (a professional term, a kind of deep learning network). The hyper-sphere center can be the center of the hyper-sphere model determined for each kind of layout pseudo label according to the labeled layout sample. Specifically, the hyper-sphere center can be determined according to the following formula:

[0038]

[0039] N is the number of marked page samples corresponding to a certain page pseudo label in the marked page samples; x i is the i-th labeled image in the marked page samples; f(x i ) is the feature of the i-th labeled image extracted by the initial feature extraction model; C is the hyper-sphere center.

[0040] S130, for each page pseudo label, based on the initial feature extraction model of the page pseudo label, according to the hyper-sphere center and the marked page samples corresponding to the page pseudo label, determine the target hyper-sphere radius of the page pseudo label.

[0041] The target hyper-sphere radius can be the radius of the hyper-sphere model corresponding to the page pseudo label, which is used to screen the candidate pseudo label samples. Specifically, the hyper-sphere radius can be obtained according to the Euclidean distance between the feature and the center. For each page pseudo label, the samples in the marked page samples with the same label as the page pseudo label are screened out as the training samples of the initial feature extraction model of the page pseudo label, and the initial feature extraction model is trained to obtain a more compact hyper-sphere model, that is, the difference between the radii corresponding to the features extracted by the initial feature extraction model of each training sample is small. According to the hyper-sphere radius corresponding to the page pseudo label, the loss function is determined, and when the loss function converges, the trained feature extraction model is obtained. According to the trained feature extraction model, all hyper-sphere radii of the corresponding marked page samples of the page pseudo label are obtained, and the target hyper-sphere radius is determined from all the obtained hyper-sphere radii. Exemplarily, the maximum radius among all the radii of the marked page samples can be taken as the target hyper-sphere radius.

[0042] S140, according to the target hyper-sphere radius of the page pseudo label, screen the target pseudo label sample from the candidate pseudo label sample corresponding to the page pseudo label.

[0043] The target pseudo label sample screening rule can be determined according to the target hyper-sphere radius of the page pseudo label. Exemplarily, the screening rule can be that if the radius of the candidate pseudo label sample is smaller than the target hyper-sphere radius, the candidate pseudo label sample is taken as the target pseudo label sample. According to the trained feature extraction model, the hyper-sphere radius of the candidate pseudo label sample corresponding to the page pseudo label can be obtained, and the obtained hyper-sphere radius is compared with the target radius. If the radius of the candidate pseudo label sample meets the target pseudo label sample screening rule, the candidate pseudo label sample is determined as the target pseudo label sample.

[0044] S150, according to the target pseudo label samples of various page pseudo labels, train the pre-trained page analysis model to obtain a target page analysis model.

[0045] The target pseudo-label sample of each layout pseudo-label is input into the pre-trained layout analysis model, the pre-trained layout analysis model updates its parameters according to the training result, that is, the pre-trained layout analysis model is updated, and a model evaluation value of the updated pre-trained layout analysis model is obtained; if the model evaluation value meets a preset model evaluation threshold, the updated pre-trained layout analysis model is determined as the target layout analysis model.

[0046] In an optional embodiment, the pre-trained layout analysis model is trained according to the target pseudo-label sample of each layout pseudo-label to obtain a target layout analysis model, including: inputting the target pseudo-label sample of each layout pseudo-label into the pre-trained layout analysis model, updating the pre-trained layout analysis model, and obtaining a model evaluation value; if the model evaluation value meets a preset model evaluation threshold, the updated pre-trained layout analysis model is determined as the target layout analysis model.

[0047] The model evaluation value can be a numerical value of an index for evaluating the performance of the model, and is used to determine whether the model is trained. For example, the model evaluation value can be at least one of accuracy, stability, and recall rate, which is not limited in the present application. The preset model evaluation threshold is a threshold of the model evaluation value, which is used to determine whether the model needs to continue training. The preset model evaluation threshold can be set according to actual needs, which is not limited in the present application. For example, the threshold of the accuracy can be 98%.

[0048] The target pseudo-label sample of each layout pseudo-label is input into the pre-trained layout analysis model, the pre-trained layout analysis model updates its parameters according to the training result, that is, the pre-trained layout analysis model is updated, and a model evaluation value of the updated pre-trained layout analysis model is obtained; if the model evaluation value meets a preset model evaluation threshold, the updated pre-trained layout analysis model is determined as the target layout analysis model. For example, if the model evaluation value is the accuracy, when the model evaluation value is greater than the preset model evaluation threshold, the updated pre-trained layout analysis model is determined as the target layout analysis model, and when the model evaluation value is less than or equal to the preset model evaluation threshold, the model continues to be trained, and the pre-trained layout analysis model is updated.

[0049] By inputting the target pseudo-label sample of each layout pseudo-label into the pre-trained layout analysis model, updating the pre-trained layout analysis model, and obtaining a model evaluation value, if the model evaluation value meets a preset model evaluation, the updated pre-trained layout analysis model is determined as the target layout analysis model, and whether the model needs to continue training is determined according to the model evaluation threshold, which can make the analysis result of the target layout analysis model meet the preset evaluation standard, provide a basis for the end of model training, and guarantee the accuracy of the target layout analysis model.

[0050] The technical scheme of the embodiment determines at least one pseudo label of a layout of an unmarked layout sample based on a pre-trained layout analysis model to obtain a candidate pseudo label sample with the pseudo label of the layout; the pre-trained layout analysis model is a deep learning model pre-trained using a marked layout sample; the pre-trained layout analysis model can be obtained by pre-training the deep learning model using the marked layout sample, and can be used to analyze the unmarked layout sample to obtain the pseudo label of the layout, thereby increasing the number of marked samples; for each type of pseudo label of the layout, a hypersphere center corresponding to the type of pseudo label of the layout is determined based on an initial feature extraction model of the type of pseudo label of the layout; for each type of pseudo label of the layout, a target hypersphere radius of the type of pseudo label of the layout is determined based on the initial feature extraction model of the type of pseudo label of the layout, according to a marked layout sample corresponding to the type of pseudo label of the layout and the hypersphere center; the hypersphere center is obtained based on the marked layout sample, and then the hypersphere radius is obtained; the accuracy of the pseudo label of the layout can be determined according to the distribution of the marked layout sample; the target pseudo label sample is selected from the candidate pseudo label sample corresponding to the type of pseudo label of the layout according to the target hypersphere radius of the type of pseudo label of the layout, thereby increasing the number of training samples of the pre-trained layout analysis model; and the pre-trained layout analysis model is trained according to the target pseudo label sample of each type of pseudo label of the layout to obtain a target layout analysis model; the data amount of the training can be increased based on the obtained target pseudo label sample, and the accuracy of the layout analysis can be improved. Therefore, the technical scheme of the present application solves the problem that deep learning requires a large number of labeled samples for supervised training, and has a high dependence on sample labels; when the number of labeled samples is small, the accuracy is low; and the effect of reducing the dependence of the deep learning model on the labeled samples and improving the accuracy of the layout analysis is achieved.

[0051] Embodiment Two

[0052] Figure 2 A flowchart of a method for determining a layout analysis model according to Embodiment Two of the present application is provided. The technical scheme of the present embodiment is further refined based on the above technical scheme.

[0053] Further, the step of "determining a target hypersphere radius of each type of pseudo label of the layout based on an initial feature extraction model of the type of pseudo label of the layout, according to a marked layout sample corresponding to the type of pseudo label of the layout and a hypersphere center" is refined as "training an initial feature extraction model corresponding to each type of pseudo label of the layout according to a marked layout sample corresponding to the type of pseudo label of the layout and the hypersphere center to obtain a target feature extraction model of the type of pseudo label of the layout; determining a candidate hypersphere radius of a candidate pseudo label sample using the target feature extraction model of the type of pseudo label of the layout; and determining a target hypersphere radius of the type of pseudo label of the layout according to the candidate hypersphere radius of the candidate pseudo label sample", so as to determine the target hypersphere radius.

[0054] Referring to Figure 2 A method for determining a layout analysis model is shown, comprising:

[0055] In S210, at least one layout pseudo-label of the unmarked layout sample is determined based on a pre-trained layout analysis model, to obtain a candidate pseudo-label sample with the layout pseudo-label.

[0056] In S220, for each type of layout pseudo-label, a hypersphere center corresponding to the type of layout pseudo-label is determined based on an initial feature extraction model of the type of layout pseudo-label.

[0057] In S230, for each type of layout pseudo-label, the initial feature extraction model corresponding to the type of layout pseudo-label is trained according to the labeled layout sample corresponding to the type of layout pseudo-label and the hypersphere center, to obtain a target feature extraction model of the type of layout pseudo-label.

[0058] The target feature extraction model is a model obtained by training the corresponding initial feature extraction model according to the labeled layout sample corresponding to the type of layout pseudo-label, and is used to determine the candidate hypersphere radius. Specifically, the corresponding feature can be obtained by inputting the labeled layout sample into the corresponding initial feature extraction model for training, the loss function is determined by the feature corresponding to the hypersphere center of the type of layout pseudo-label, the parameters of the initial feature extraction model are adjusted according to the loss function, and the target feature extraction model is obtained.

[0059] In an optional embodiment, training the initial feature extraction model corresponding to the type of layout pseudo-label according to the labeled layout sample corresponding to the type of layout pseudo-label and the hypersphere center to obtain the target feature extraction model of the type of layout pseudo-label comprises: extracting the feature of the labeled layout sample by using the initial feature extraction model corresponding to the type of layout pseudo-label; determining the distance loss value according to the feature of the labeled layout sample and the hypersphere center; adjusting the model parameters of the initial feature extraction model according to the distance loss value to obtain the trained target feature extraction model.

[0060] The labeled layout sample corresponding to the type of layout pseudo-label is input into the corresponding initial feature extraction model to obtain the feature of the labeled layout sample. The distance loss value is determined according to the distance value between the feature of the labeled layout sample and the hypersphere center. For example, the distance loss value can be determined according to the Euclidean distance between the feature of the labeled layout sample and the hypersphere center. The model parameters of the initial feature extraction model are adjusted according to the distance loss value through the error backpropagation mechanism of the deep learning network to obtain the trained target feature extraction model.

[0061] The initial feature extraction model corresponding to the layout pseudo label is adopted to extract features of the marked layout sample; a distance loss value is determined according to the features of the marked layout sample and the hypersphere center; and the model parameters of the initial feature extraction model are adjusted according to the distance loss value to obtain a trained target feature extraction model, so that more similar features can be obtained through the target feature extraction model, and the accuracy of determining the target hypersphere radius corresponding to various layout pseudo labels is improved.

[0062] In an optional embodiment, the distance loss value is determined according to the distance between the features of the candidate pseudo label sample and the hypersphere center.

[0063] The distance loss value is the sum of the Euclidean distances between all the features of the marked layout sample corresponding to the layout pseudo label and the hypersphere center. Specifically, the distance loss value can be represented by the following formula:

[0064]

[0065] wherein x i The i-th sample in the marked layout sample corresponding to the layout pseudo label; f(x i ) is the i-th feature extracted by the initial feature extraction model; c is the hypersphere center; and Loss h is the distance loss.

[0066] By determining the distance loss value according to the distance between the features of the candidate pseudo label sample and the hypersphere center, the hypersphere model determined by the distance loss value can be more compact, and the accuracy of the subsequent target hypersphere radius is improved.

[0067] S240, a target feature extraction model of the layout pseudo label is adopted to determine a candidate hypersphere radius according to the corresponding marked layout sample.

[0068] The corresponding features are obtained by inputting the marked layout sample corresponding to the layout pseudo label into the corresponding target feature extraction model, and the candidate hypersphere radius of each corresponding marked layout sample is obtained by calculating the distance between the features and the hypersphere center.

[0069] S250, a target hypersphere radius of the layout pseudo label is determined according to the candidate hypersphere radius.

[0070] The candidate hypersphere radius is compared to determine the target hypersphere radius of the layout pseudo label. For example, the maximum value of the candidate hypersphere radius of the same layout pseudo label can be determined as the target hypersphere radius of the layout pseudo label.

[0071] In an optional embodiment, the target hypersphere radius of the type of layout pseudo-label is determined according to the candidate hypersphere radius, including: taking the maximum value in the candidate hypersphere radius as the target hypersphere radius of the type of layout pseudo-label.

[0072] Taking the maximum value in the candidate hypersphere radius as the target hypersphere radius of the type of layout pseudo-label can obtain as many target pseudo-label samples as possible within the error tolerance range.

[0073] S260, screening target pseudo-label samples from the candidate pseudo-label samples corresponding to the type of layout pseudo-label according to the target hypersphere radius of the type of layout pseudo-label.

[0074] In an optional embodiment, the target pseudo-label samples are screened from the candidate pseudo-label samples corresponding to the type of layout pseudo-label according to the target hypersphere radius of the type of layout pseudo-label, including: determining the candidate pseudo-label radius of each candidate pseudo-label sample based on the target feature extraction model of the type of layout pseudo-label; if the candidate pseudo-label radius is smaller than the target hypersphere radius, the candidate pseudo-label sample corresponding to the candidate pseudo-label radius is taken as the target pseudo-label sample.

[0075] Each candidate pseudo-label sample is input into the target feature extraction model of the type of layout pseudo-label, and the candidate pseudo-label radius of each candidate pseudo-label sample is obtained by calculating the distance between the extracted feature and the hypersphere center. If the candidate pseudo-label radius is smaller than the target hypersphere radius, the candidate pseudo-label sample corresponding to the candidate pseudo-label radius is taken as the target pseudo-label sample.

[0076] By determining the candidate pseudo-label radius of each candidate pseudo-label sample based on the target feature extraction model of the type of layout pseudo-label, if the candidate pseudo-label radius is smaller than the target hypersphere radius, the candidate pseudo-label sample corresponding to the candidate pseudo-label radius is taken as the target pseudo-label sample, the target pseudo-label sample is screened from the candidate pseudo-label sample, and the accuracy of the label of the target pseudo-label sample is improved.

[0077] S270, training the pre-trained layout analysis model according to the target pseudo-label samples of various types of layout pseudo-labels to obtain a target layout analysis model.

[0078] The technical scheme of the embodiment trains the initial feature extraction model corresponding to each kind of layout pseudo label according to the marked layout sample corresponding to the layout pseudo label and the hypersphere center, obtains the target feature extraction model of the layout pseudo label, and improves the feature extraction accuracy of the target feature extraction model; the target feature extraction model of the layout pseudo label is used to determine the candidate hypersphere radius according to the corresponding marked layout sample; and the target hypersphere radius of the layout pseudo label is determined according to the candidate hypersphere radius, so that the target hypersphere radius can be determined through the marked layout sample, the accuracy of the target hypersphere radius can be improved, and the accuracy of the target pseudo label sample can be improved.

[0079] Embodiment three

[0080] Figure 3 Fig. 3 shows a structural schematic diagram of a layout analysis model determination device provided by Embodiment Three of the present application. The embodiment can be applied to the case where a deep learning network is trained through fewer label samples so that the trained deep learning network can accurately perform layout analysis on a layout image. The specific structure of the layout analysis model determination device is as follows:

[0081] The layout pseudo label marking module 310 is configured to determine at least one layout pseudo label of an unmarked layout sample based on a pre-trained layout analysis model, and obtain a candidate pseudo label sample with the layout pseudo label. The pre-trained layout analysis model is a deep learning model pre-trained using a marked layout sample.

[0082] The hypersphere center determination module 320 is configured to determine, for each kind of layout pseudo label, a hypersphere center corresponding to the layout pseudo label based on an initial feature extraction model of the layout pseudo label.

[0083] The target hypersphere radius determination module 330 is configured to determine, for each kind of layout pseudo label, a target hypersphere radius of the layout pseudo label based on an initial feature extraction model of the layout pseudo label and a marked layout sample corresponding to the layout pseudo label and a hypersphere center.

[0084] The target pseudo label sample screening module 340 is configured to screen a target pseudo label sample from the candidate pseudo label sample corresponding to the layout pseudo label according to the target hypersphere radius of the layout pseudo label.

[0085] The target layout analysis model determination module 350 is configured to train the pre-trained layout analysis model according to the target pseudo label sample of each kind of layout pseudo label, and obtain a target layout analysis model.

[0086] The technical scheme of the embodiment determines at least one layout pseudo label of an unmarked layout sample based on a pre-trained layout analysis model to obtain a candidate pseudo label sample with the layout pseudo label; the pre-trained layout analysis model is a deep learning model pre-trained using marked layout samples; the pre-trained layout analysis model can be obtained by pre-training the deep learning model using the marked layout samples, and can be used to analyze the unmarked layout sample to obtain the layout pseudo label, thereby increasing the number of marked samples; for each type of layout pseudo label, an initial feature extraction model based on the type of layout pseudo label is used to determine a hyper-spherical ball center corresponding to the type of layout pseudo label; for each type of layout pseudo label, an initial feature extraction model based on the type of layout pseudo label is used to determine a target hyper-spherical radius of the type of layout pseudo label according to the marked layout sample corresponding to the type of layout pseudo label and the hyper-spherical ball center; the hyper-spherical ball center is obtained based on the marked layout sample, and then the hyper-spherical radius is obtained; the accuracy of the layout pseudo label can be determined according to the distribution of the marked layout sample; the target pseudo label sample is selected from the candidate pseudo label sample corresponding to the type of layout pseudo label according to the target hyper-spherical radius of the type of layout pseudo label, thereby increasing the number of training samples of the pre-trained layout analysis model; the pre-trained layout analysis model is trained according to the target pseudo label sample of each type of layout pseudo label to obtain a target layout analysis model; the data volume of the training can be increased based on the obtained target pseudo label sample, and the accuracy of the layout analysis can be improved. Therefore, the technical scheme of the present application solves the problem that deep learning requires a large number of labeled samples for supervised training, and has a high dependence on sample labels; when the number of labeled samples is small, the accuracy is low; the dependence of the deep learning model on the labeled samples is reduced, and the accuracy of the layout analysis is improved.

[0087] Optionally, the target hyper-spherical radius determination module 330 comprises:

[0088] The target feature extraction model determination unit is configured to train the initial feature extraction model corresponding to each type of layout pseudo label according to the marked layout sample corresponding to the type of layout pseudo label and the hyper-spherical ball center to obtain the target feature extraction model of the type of layout pseudo label.

[0089] The candidate hyper-spherical radius determination unit is configured to determine the candidate hyper-spherical radius according to the corresponding marked layout sample using the target feature extraction model of the type of layout pseudo label.

[0090] The target hyper-spherical radius determination unit is configured to determine the target hyper-spherical radius of the type of layout pseudo label according to the candidate hyper-spherical radius.

[0091] Optionally, the target feature extraction model determination unit comprises:

[0092] The feature extraction subunit is configured to extract features of the marked layout sample by using an initial feature extraction model corresponding to the layout pseudo label.

[0093] The distance loss value determination subunit is configured to determine a distance loss value according to the features of the marked layout sample and the hypersphere center.

[0094] The model parameter adjustment subunit is configured to adjust model parameters of the initial feature extraction model according to the distance loss value, to obtain a trained target feature extraction model.

[0095] Optionally, the distance loss value determination subunit is specifically configured to:

[0096] determine the distance loss value according to a distance between the features of the candidate pseudo label sample and the hypersphere center.

[0097] Optionally, the target hypersphere radius determination unit comprises:

[0098] The candidate hypersphere radius comparison subunit is configured to take a maximum value in the candidate hypersphere radii as the target hypersphere radius of the layout pseudo label.

[0099] Optionally, the target pseudo label sample screening module 340 comprises:

[0100] The candidate pseudo label radius determination unit is configured to determine candidate pseudo label radii of the candidate pseudo label samples based on the target feature extraction model of the layout pseudo label.

[0101] The target pseudo label sample determination unit is configured to take a candidate pseudo label sample corresponding to a candidate pseudo label radius as a target pseudo label sample if the candidate pseudo label radius is smaller than the target hypersphere radius.

[0102] Optionally, the target layout analysis model determination module 350 comprises:

[0103] The layout analysis model training unit is configured to input the target pseudo label samples of various layout pseudo labels into the pre-trained layout analysis model, update the pre-trained layout analysis model, and obtain a model evaluation value.

[0104] The target layout analysis model determination unit is configured to take the updated pre-trained layout analysis model as a target layout analysis model if the model evaluation value meets a preset model evaluation threshold.

[0105] The layout analysis model determination apparatus provided in the embodiments of the present application can perform the layout analysis model determination method provided in any of the embodiments of the present application, and has the corresponding function modules and beneficial effects of performing the layout analysis model determination method.

[0106] Embodiment Four

[0107] Figure 4 FIG. 4 is a structural schematic diagram of an electronic device according to an embodiment of the present application. Figure 4 As shown in FIG. 4, the electronic device includes a processor 410, a memory 420, an input device 430, and an output device 440; the number of processors 410 in the electronic device can be one or more, and one processor 410 is taken as an example in the embodiment. Figure 4 The processor 410, the memory 420, the input device 430, and the output device 440 in the electronic device can be connected through a bus or other means. Figure 4 The connection through the bus is taken as an example.

[0108] The memory 420, as a computer-readable storage medium, can be used to store software programs, computer-executable programs, and modules, such as program instructions / modules of the determination method of the layout analysis model (for example, the layout pseudo-label marking module 310, the hyper-sphere center determination module 320, the target hyper-sphere radius determination module 330, the target pseudo-label sample screening module 340, and the target layout analysis model determination module 350) in the embodiment of the present application. The processor 410 executes various function applications and data processing of the electronic device by running the software programs, instructions, and modules stored in the memory 420, that is, implements the above-mentioned determination method of the layout analysis model.

[0109] The memory 420 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system and application programs required by at least one function; the data storage area can store data created according to the use of the terminal, etc. In addition, the memory 420 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other non-volatile solid-state memory device. In some examples, the memory 420 can further include a memory remotely arranged with respect to the processor 410, which can be connected to the electronic device through a network. Examples of the above-mentioned network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0110] The input device 430 can be used to receive input character information, and generate key signal input related to user settings and function control of the electronic device. The output device 440 can include a display device such as a display screen.

[0111] Embodiment five

[0112] The embodiment five of the present application further provides a storage medium comprising computer executable instructions, which, when executed by a computer processor, are used to perform a method for determining a layout analysis model, the method comprising: determining at least one layout pseudo label of an unmarked layout sample based on a pre-trained layout analysis model to obtain a candidate pseudo label sample with the layout pseudo label; the pre-trained layout analysis model being a deep learning model pre-trained by using marked layout samples; for each kind of layout pseudo label, determining a hyper-sphere center corresponding to the kind of layout pseudo label based on an initial feature extraction model of the kind of layout pseudo label; for each kind of layout pseudo label, determining a target hyper-sphere radius of the kind of layout pseudo label based on the initial feature extraction model of the kind of layout pseudo label and according to a marked layout sample corresponding to the kind of layout pseudo label and the hyper-sphere center; and screening a target pseudo label sample from the candidate pseudo label sample corresponding to the kind of layout pseudo label according to the target hyper-sphere radius of the kind of layout pseudo label; and training the pre-trained layout analysis model according to the target pseudo label samples of various kinds of layout pseudo labels to obtain a target layout analysis model.

[0113] Of course, the storage medium comprising computer executable instructions provided by the embodiment of the present application is not limited to the method operations described above, and can also perform related operations in the method for determining a layout analysis model provided by any embodiment of the present application.

[0114] From the above description of the embodiments, those skilled in the art can clearly understand that the present application can be implemented by means of software and necessary general hardware, and of course can also be implemented by hardware, but in many cases the former is a better embodiment. Based on such understanding, the technical solutions of the present application can be embodied in the form of a software product, which can be stored in a computer readable storage medium, such as a floppy disk, a read-only memory (ROM), a random access memory (RAM), a FLASH memory, a hard disk or an optical disk, and includes a number of instructions to make an electronic device (which can be a personal computer, a server, or a network device, etc.) execute the methods described in various embodiments of the present application.

[0115] It is worth noting that in the above embodiment of the search device, each unit and module included is only divided according to functional logic, but is not limited to the above division, as long as the corresponding functions can be realized; in addition, the specific names of each functional unit are only for easy mutual distinction, and do not limit the protection scope of the present application.

[0116] It is to be noted that the above-mentioned embodiments illustrate rather than limit the application, and that those skilled in the art will be able to design many alternative embodiments without departing from the scope of the application. The word "comprising" does not exclude the presence of elements or steps other than those listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. It is further noted that characteristics relating to the different embodiments can be combined, and not just those within respective sections of the description.

Claims

1. A method for determining a layout analysis model, characterized in that, include: Based on the pre-trained layout analysis model, at least one pseudo-label of an unlabeled layout sample is determined, and candidate pseudo-label samples with the pseudo-label are obtained. The pre-trained layout analysis model is a deep learning model pre-trained using labeled layout samples; For each type of pseudo-tag, the center of the hypersphere corresponding to that type of pseudo-tag is determined based on the initial feature extraction model of that type of pseudo-tag. For each type of pseudo-label, based on the initial feature extraction model of that type of pseudo-label, the target hypersphere radius of that pseudo-label is determined according to the labeled page sample corresponding to that type of pseudo-label and the center of the hypersphere. Based on the target hypersphere radius of this type of pseudo-label, select target pseudo-label samples from the candidate pseudo-label samples corresponding to this type of pseudo-label. The pre-trained layout analysis model is trained using target pseudo-label samples of various layout pseudo-labels to obtain the target layout analysis model.

2. The method according to claim 1, characterized in that, For each type of page layout pseudo-label, based on the initial feature extraction model of that type of page layout pseudo-label, and according to the marked page sample corresponding to that type of page layout pseudo-label and the center of the hypersphere, the radius of the target hypersphere of that page layout pseudo-label is determined, including: For each type of pseudo-label, based on the labeled page sample corresponding to the pseudo-label and the center of the hypersphere, the initial feature extraction model corresponding to the pseudo-label is trained to obtain the target feature extraction model of the pseudo-label. The target feature extraction model of this type of page pseudo-label is used to determine the radius of the candidate hypersphere based on the corresponding labeled page samples; Based on the candidate hypersphere radius, the target hypersphere radius of this type of layout pseudo-label is determined.

3. The method according to claim 2, characterized in that, The step of training the initial feature extraction model corresponding to the pseudo-label based on the marked page sample and the center of the hypersphere to obtain the target feature extraction model for the pseudo-label includes: The features of the marked page sample are extracted using the initial feature extraction model corresponding to this type of page pseudo-label. Based on the characteristics of the marked sample and the center of the hypersphere, the distance loss value is determined; Based on the distance loss value, the model parameters of the initial feature extraction model are adjusted to obtain the trained target feature extraction model.

4. The method according to claim 3, characterized in that, The step of determining the distance loss value based on the features of the candidate pseudo-label samples and the center of the hypersphere includes: The distance loss value is determined based on the characteristics of the candidate pseudo-label samples and the distance to the center of the hypersphere.

5. The method according to claim 2, characterized in that, The determination of the target hypersphere radius of the pseudo-label based on the radius of each candidate hypersphere includes: The maximum value among the candidate hypersphere radii is taken as the target hypersphere radius of this type of layout pseudo-label.

6. The method according to claim 2, characterized in that, The step of selecting target pseudo-label samples from the candidate pseudo-label samples corresponding to this type of layout pseudo-label based on the target hypersphere radius of this type of layout pseudo-label includes: Based on the target feature extraction model of this type of page pseudo-label, the candidate pseudo-label radius of each candidate pseudo-label sample is determined; If the radius of the candidate pseudo-label is smaller than the radius of the target hypersphere, then the candidate pseudo-label sample corresponding to the radius of the candidate pseudo-label is taken as the target pseudo-label sample.

7. The method according to claim 1, characterized in that, The step of training the pre-trained layout analysis model based on target pseudo-tag samples of various layout pseudo-tags to obtain the target layout analysis model includes: The target pseudo-label samples of the various layout pseudo-labels are input into the pre-trained layout analysis model to update the pre-trained layout analysis model and obtain the model evaluation value. If the model evaluation value meets the preset model evaluation threshold, the updated pre-trained layout analysis model will be used as the target layout analysis model.

8. A device for determining a page layout analysis model, characterized in that, include: The layout pseudo-labeling module is used to determine at least one layout pseudo-label for unlabeled layout samples based on a pre-trained layout analysis model, and to obtain candidate pseudo-label samples with the layout pseudo-label; the pre-trained layout analysis model is a deep learning model pre-trained using labeled layout samples; The hypersphere center determination module is used to determine the hypersphere center corresponding to each type of pseudo-label based on the initial feature extraction model of that type of pseudo-label. The target hypersphere radius determination module is used to determine the target hypersphere radius of each type of pseudo-label based on the initial feature extraction model of that type of pseudo-label, according to the marked page sample corresponding to that type of pseudo-label and the center of the hypersphere. The target pseudo-label sample filtering module is used to filter target pseudo-label samples from the candidate pseudo-label samples corresponding to the type of layout pseudo-label based on the target hypersphere radius of the pseudo-label. The target layout analysis model determination module is used to train the pre-trained layout analysis model based on target pseudo-label samples of various layout pseudo-labels to obtain the target layout analysis model.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the method for determining the layout analysis model as described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by the processor, the program implements the method for determining the layout analysis model as described in any one of claims 1-7.

Images