A model training method, device, storage medium and electronic device

Abstract

The specification provides a model training method, device, storage medium and electronic equipment. By setting different training tasks, including restoring the applet page, predicting the service type of the applet and the element type of the applet page, etc., the applet recognition model not only has the recognition ability of text and image, but also has the understanding ability of the layout mode of the applet page, thereby improving the training effect of the model. The specification also provides a method for recognizing an applet, using an applet recognition model trained by the above-mentioned model training method, recognizing the applet, extracting the features of the applet according to the image corresponding to the page of the applet, and using the features to achieve accurate recognition of the applet, thereby improving the efficiency of recognizing the applet.

Description

Technical Field

[0001] This application relates to the field of computers, and in particular to a method, apparatus, storage medium, and electronic device for model training. Background Technology

[0002] In the information age, many businesses provide services to users through mini-programs on the Internet. In order to ensure the health and security of the Internet, it is often necessary to identify whether the services provided by these mini-programs are legal. If it is time-consuming and laborious to entrust these identification tasks to humans, then it is convenient and fast to entrust these tasks to computers.

[0003] To facilitate the identification of the large and diverse number of mini-programs, the internet typically requires merchants to submit application materials for the mini-program. The legitimacy of the services provided by the mini-program is then determined based on these materials. However, some merchants may engage in deception or misrepresentation for personal gain. For example, if the application materials indicate that the merchant's mini-program sells apples, the product information within the mini-program might be changed to an Apple phone, causing users to purchase an Apple product at the price of an Apple phone. In such cases, the internet needs to possess the capability to independently identify mini-programs.

[0004] Therefore, this application proposes a method for identifying mini-programs by training a model. Summary of the Invention

[0005] This specification provides a method, apparatus, and storage medium for model training, as well as a method, apparatus, storage medium, and electronic device for identifying applets, to at least partially solve the above-mentioned problems.

[0006] The following technical solution is adopted in this specification:

[0007] This specification provides a method for model training, the method comprising:

[0008] Identify the images corresponding to the pages of the mini-program and use them as training samples;

[0009] The training samples are input into the feature extraction subnet of the mini-program recognition model to obtain the features of the training samples output by the feature extraction subnet.

[0010] At least a portion of the features are input into the prediction subnet of the mini-program recognition model to obtain the prediction results output by the prediction subnet. The prediction results include at least one of the following: the page restoration result, the prediction result of the service type provided by the mini-program, and the prediction result of the type of each element contained in the page. The page restoration result is the information corresponding to the part of the features that is not input into the prediction subnet in the page.

[0011] The final loss is determined based on the prediction results and the labels corresponding to the training samples;

[0012] The mini-program recognition model is trained with the goal of minimizing the final loss.

[0013] Optionally, obtaining the features of the training samples output by the feature extraction subnet specifically includes:

[0014] The feature extraction subnetwork is used to extract features from the images in the training samples to obtain the image features output by the feature extraction subnetwork.

[0015] The feature extraction subnetwork is used to extract features from the text in the training samples to obtain the text features output by the feature extraction subnetwork.

[0016] The image features and text features are fused to obtain the features of the training sample.

[0017] Optionally, feature extraction is performed on the images in the training samples, specifically including:

[0018] At least a portion of the images in the training samples are removed, and features are extracted from the remaining images in the training samples.

[0019] Feature extraction is performed on the text in the training samples, specifically including:

[0020] At least a portion of the text in the training samples is removed, and features are extracted from the remaining text in the training samples.

[0021] Optionally, when the prediction result includes prediction results for the types of each element contained in the page, the final loss is determined based on the prediction result and the annotations corresponding to the training samples, specifically including:

[0022] The training samples are input into the layout analysis model to obtain the type of each element in the training samples output by the layout analysis model, which is then used as a label.

[0023] The final loss is determined based on the prediction results of the types of each element contained in the page output by the prediction subnet and the annotations.

[0024] Optionally, when the prediction result includes the page reconstruction result, the final loss is determined based on the prediction result and the annotations corresponding to the training samples, specifically including:

[0025] The image corresponding to the page is used as a label, and the final loss is determined based on the page reconstruction result output by the prediction subnet and the label.

[0026] Optionally, when the prediction result includes the prediction result of the service type provided by the mini-program, the final loss is determined based on the prediction result and the labels corresponding to the training samples, specifically including:

[0027] The service provider for the mini-program is identified and the service type preset for the mini-program is used as a label;

[0028] The final loss is determined based on the prediction results of the service type provided by the mini-program output by the prediction subnet and the annotation.

[0029] Optionally, the method further includes:

[0030] The features of the training samples output by the feature extraction subnet are input into the recognition subnet of the mini-program recognition model, so that the standard type label corresponding to the service provided by the mini-program can be determined from the preset standard type labels through the recognition subnet.

[0031] Based on the standard type labels output by the recognition subnet and the annotations of the training samples, the recognition loss of the mini-program recognition model is determined;

[0032] The mini-program recognition model is trained with the goal of minimizing the recognition loss.

[0033] This specification also provides a method for identifying mini-programs, the method comprising:

[0034] Identify the image corresponding to the page of the mini-program to be identified, and use it as the image to be identified;

[0035] The image to be identified is input into the feature extraction subnet of the mini-program recognition model to obtain the features of the image to be identified output by the feature extraction subnet;

[0036] The features are input into the recognition subnet of the mini-program recognition model to obtain the recognition result output by the recognition subnet. The mini-program recognition model is trained using the model training method described above.

[0037] This specification also provides a model training apparatus, the apparatus comprising:

[0038] The sample extraction module is used to determine the images corresponding to the pages of the mini-program as training samples.

[0039] The extraction module is used to input the training samples into the feature extraction subnet in the mini-program recognition model to obtain the features of the training samples output by the feature extraction subnet.

[0040] The prediction module is used to input at least a portion of the features into the prediction subnet of the mini-program recognition model to obtain the prediction result output by the prediction subnet. The prediction result includes at least one of the following: the page restoration result, the prediction result of the service type provided by the mini-program, and the prediction result of the type of each element contained in the page. The page restoration result is the information corresponding to the part of the features that was not input into the prediction subnet in the page.

[0041] The loss determination module is used to determine the final loss based on the prediction results and the labels corresponding to the training samples;

[0042] The training module is used to train the mini-program recognition model with the goal of minimizing the final loss.

[0043] This specification also provides a device for identifying mini-programs, the device comprising:

[0044] The input module is used to determine the image corresponding to the page of the mini-program to be identified, which is then used as the image to be identified.

[0045] The feature extraction module is used to input the image to be recognized into the feature extraction subnet in the mini-program recognition model to obtain the features of the image to be recognized output by the feature extraction subnet;

[0046] The recognition module is used to input the features into the recognition subnet of the mini-program recognition model and obtain the recognition result output by the recognition subnet. The mini-program recognition model is trained using the model training method described above.

[0047] This specification provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method for training the model and the method for recognizing the applet.

[0048] This specification provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the above-described method for training the model and the method for recognizing the applet.

[0049] The above-mentioned technical solutions adopted in this specification can achieve the following beneficial effects:

[0050] In the model training method provided in this manual, by setting different training tasks, including restoring mini-program pages, predicting the service type of mini-programs, and the element type of mini-program pages, the mini-program recognition model not only has the ability to recognize text and images, but also the ability to understand the layout of mini-program pages, thereby improving the training effect of the model.

[0051] This manual also provides a method for recognizing mini-programs. The mini-program recognition model trained using the above-mentioned model training method is used to recognize mini-programs. The features of the mini-program are extracted from the image corresponding to the page of the mini-program, and the mini-program is accurately recognized by using these features, thereby improving the efficiency of mini-program recognition. Attached Figure Description

[0052] The accompanying drawings, which are included to provide a further understanding of this specification and form part of this specification, illustrate exemplary embodiments and are used to explain this specification, but do not constitute an undue limitation thereof. In the drawings:

[0053] Figure 1 This is a flowchart illustrating one model training method described in this specification.

[0054] Figure 2 A schematic diagram illustrating the prediction result types for the prediction subnets provided in this specification;

[0055] Figure 3 This is a schematic diagram of a recognition model using a mini-program as described in this specification;

[0056] Figure 4 This is a schematic diagram illustrating the identification of a mini-program as provided in this specification.

[0057] Figure 5 The corresponding information provided in this specification Figure 1 A schematic diagram of an electronic device;

[0058] Figure 6 This is a schematic diagram of a model training device provided in this specification;

[0059] Figure 7 The corresponding information provided in this specification Figure 1 A schematic diagram of an electronic device. Detailed Implementation

[0060] To make the objectives, technical solutions, and advantages of this specification clearer, the technical solutions of this specification will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this specification, and not all of them. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0061] The technical solutions provided in the various embodiments of this specification are described in detail below with reference to the accompanying drawings.

[0062] Figure 1 This document provides a flowchart illustrating a model training method, which includes the following steps:

[0063] S100: Determine the image corresponding to the page of the mini-program and use it as a training sample.

[0064] The execution subject for model training provided in this manual can be a server or an electronic device such as a personal computer (PC) or mobile phone. For ease of description, the following description will only use a server as the execution subject to explain the model training method provided in this manual.

[0065] In the embodiments of this specification, images corresponding to the pages of the mini-program are selected as training samples. This can train the text recognition ability of the mini-program recognition model, as well as its ability to understand images and page layouts. The images corresponding to the pages of the mini-program are screenshots of the mini-program pages. The number of training samples can be determined according to specific circumstances, and no limit is placed on the number of training samples in this specification.

[0066] S101: Input the training samples into the feature extraction subnet of the mini-program recognition model to obtain the features of the training samples output by the feature extraction subnet.

[0067] For the training samples input into the feature extraction subnet, the feature extraction subnet extracts features from the text in the training samples. Specifically, the text in the training samples can first be recognized by optical character recognition (OCR), and for each recognized character, the character is input into the feature extraction subnet to extract the features corresponding to the character. Then, based on the extracted features corresponding to each character, the text features of the training sample are determined.

[0068] For the training samples input into the feature extraction subnet, the feature extraction subnet extracts features from the images in the training samples. Specifically, for each image in the training samples, the image can be scaled to a specified size, and then the scaled image can be input into the feature extraction subnet to extract the features corresponding to the image. Then, based on the extracted features corresponding to each image, the image features of the training sample are determined.

[0069] When scaling the image to a specified size, the image is first scaled to a specified width or height. If the image is scaled to a specified width, the height of the scaled image is checked to see if it exceeds the preset size. If it does, the excess portion is cropped; if it does not exceed the preset size, the blank portion is added. If the image is scaled to a specified height, the width of the scaled image is checked to see if it exceeds the preset size. If it does, the excess portion is cropped; if it does not exceed the preset size, the blank portion is added.

[0070] After obtaining the text features and image features of the training samples, the text features and image features can be fused together and used as the features of the training samples.

[0071] S102: Input at least a portion of the features into the prediction subnet of the mini-program recognition model to obtain the prediction result output by the prediction subnet. The prediction result includes at least one of the following: the page restoration result, the prediction result of the service type provided by the mini-program, and the prediction result of the type of each element contained in the page. The page restoration result is the information corresponding to the part of the features that was not input into the prediction subnet in the page.

[0072] like Figure 2 As shown, when predicting the service type of a mini-program and the types of each element on the page are used as training tasks, all the features of the training samples extracted by the feature extraction subnet can be input into the prediction subnet of the mini-program recognition model.

[0073] When using the page restoration result as a training task, a portion of the features extracted from the training samples by the feature extraction subnet can be input into the prediction subnet of the mini-program recognition model.

[0074] Specifically, after extracting text features from the training samples, the text features are selected, and the selected parts are removed. The remaining text features are input into the prediction subnetwork, which outputs the text reconstruction result of the page. The text reconstruction result of the page is compared with the text in the image corresponding to the page of the mini-program to determine the loss. And / or, after extracting image features from the training samples, the image features are selected, and the selected image blocks are removed. The remaining image blocks are used as part of the image features of the training samples and input into the prediction subnetwork. The prediction subnetwork outputs the image reconstruction result of the page. The image reconstruction result of the page is compared with the image corresponding to the page of the mini-program to determine the loss. Here, the removal can be processed by a mask matrix on the text features, or on the image features, or on both the text features and the image features.

[0075] S103: Determine the final loss based on the prediction results and the labels corresponding to the training samples.

[0076] When the prediction result is the text reconstruction result of the mini-program page, the text in the image corresponding to the mini-program page can be used as the annotation of the training sample, and the reconstruction result is compared with the annotation to determine the difference between the prediction result and the annotation, thereby determining the first loss.

[0077] When the prediction result is the image reconstruction result of the mini-program page, the image corresponding to the mini-program page can be used as the annotation of the training sample, and the reconstruction result is compared with the annotation to determine the difference between the prediction result and the annotation, thereby determining the second loss.

[0078] In the embodiments of this specification, the trained mini-program recognition model is used to determine which type of label among the preset standard type labels the mini-program to be recognized belongs to based on the screenshot of the mini-program page. However, directly comparing the preset standard type label as the annotation with the prediction result will result in a large difference between the two, which will lead to poor training effect. Therefore, when training the mini-program recognition model, the service type of the mini-program provided by the service provider can be used as the annotation of the training sample. When the prediction result is the service type provided by the mini-program, the difference between the prediction result and the annotation is determined, thereby determining the third loss.

[0079] When the prediction result is the type of each element contained in the mini-program page, a distillation method can be used. The pre-trained layout analysis model is used as the teacher model, and the mini-program recognition model is used as the student model. The same content is input, and the output result of the teacher model is used as the training target of the student model. Therefore, the training samples are input into the layout analysis model and the mini-program recognition model respectively. The output result of the layout analysis model is used as the label of the training samples. The output result of the mini-program recognition model is compared with the label to determine the difference between the prediction result and the label, thereby determining the fourth loss. Here, the layout analysis model is a model that has been trained and can output accurate results, and the fourth loss can be the cross-entropy between the prediction result and the label.

[0080] After determining the first loss, second loss, third loss, and fourth loss, at least one of the first loss, second loss, third loss, and fourth loss is weighted to determine the final loss.

[0081] S104: Train the mini-program recognition model with the goal of minimizing the final loss.

[0082] In the embodiments of this specification, the mini-program recognition model is trained through different training tasks, so that the trained mini-program can accurately extract the features of the page screenshot based on the page screenshot of the mini-program to be recognized, and then determine which system preset standard type label the mini-program to be recognized belongs to. Therefore, enabling the mini-program recognition model to accurately extract the features of the page screenshot is the key to training the mini-program recognition model. That is, when training the mini-program recognition model, the parameters of the feature extraction subnet in the mini-program recognition model can be adjusted.

[0083] In the model training method provided in this specification, the images of the acquired mini-program pages are used as training samples for feature extraction. The features include text and image features, and the features are input into the prediction subnet of the mini-program recognition model to obtain the prediction results. The labels of the training samples are preset, and the labels are compared with the prediction results to determine the final loss and minimize the final loss. The above method enables the mini-program training model to have the ability of text recognition and image understanding, thereby achieving accurate recognition of mini-programs and improving the efficiency of mini-program recognition.

[0084] In the embodiments of this specification, after training the mini-program recognition model, it is necessary to test the training effect of the model and fine-tune the model based on the test results. The model structure in actual use is different from the model structure during training, such as... Figure 3 As shown, an identification subnet can be used to replace it. Figure 2 The prediction subnet in the model structure shown specifically uses the page image of the mini-program as a training sample and inputs it into the feature extraction subnet of the model for feature extraction. The extracted features are then input into the recognition subnet of the model. Based on preset standard type labels, the model determines which standard type label the service provided by the mini-program belongs to, and uses the determined result as the label of the training sample. The model compares the output of the recognition subnet with the label to determine the difference between the output of the recognition subnet and the label, and then determines the recognition loss. The model is adjusted with the goal of minimizing the recognition loss. Specifically, the model parameters of the feature extraction subnet and the recognition subnet can be adjusted when adjusting the model.

[0085] Accordingly, adopting such Figure 1 The method shown was used to train the mini-program recognition model, and the above method was applied to programs such as... Figure 3 After fine-tuning the mini-program recognition model shown, the method for using the mini-program recognition model is as follows: Figure 4 As shown, the specific steps are as follows:

[0086] S400: Determine the image corresponding to the page of the mini-program to be identified, and use it as the image to be identified.

[0087] Take a screenshot of the page of the mini-program to be identified, and use the screenshot as the image to be identified.

[0088] S401: Input the image to be recognized into the feature extraction subnet in the mini-program recognition model to obtain the features of the image to be recognized output by the feature extraction subnet.

[0089] The trained mini-program recognition model inputs the image to be recognized into the feature extraction subnet of the mini-program recognition model to obtain the features of the image to be recognized, wherein the features of the image to be recognized are the text features and / or image features of the image to be recognized.

[0090] S402: Input the features into the recognition subnet of the mini-program recognition model to obtain the recognition result output by the recognition subnet.

[0091] After obtaining the features of the image to be identified, the features of the image to be identified are input into the recognition subnet of the mini-program recognition model to obtain the recognition result of the mini-program to be identified, that is, which preset standard type label the mini-program to be identified belongs to.

[0092] The method for identifying mini-programs provided in this specification extracts features from screenshots of mini-program pages. This extraction process can capture both text and image features of the mini-program page, or both simultaneously. Based on the extracted features, the method determines which preset standard type tag the mini-program belongs to, thereby improving the efficiency and accuracy of mini-program identification.

[0093] The above describes one or more embodiments of the model training method provided in this specification. Based on the same idea, this specification also provides corresponding model training devices, such as... Figure 5 As shown.

[0094] Figure 5 A schematic diagram of a model training device provided in this specification specifically includes:

[0095] The sample extraction module 501 is used to determine the image corresponding to the page of the mini program as a training sample;

[0096] Extraction module 502 is used to input the training sample into the feature extraction subnet in the mini-program recognition model to obtain the features of the training sample output by the feature extraction subnet;

[0097] The prediction module 503 is used to input at least a portion of the features into the prediction subnet of the mini-program recognition model to obtain the prediction result output by the prediction subnet. The prediction result includes at least one of the following: the page restoration result, the prediction result of the service type provided by the mini-program, and the prediction result of the type of each element contained in the page. The page restoration result is the information corresponding to the part of the features that was not input into the prediction subnet in the page.

[0098] The loss determination module 504 is used to determine the final loss based on the prediction result and the labels corresponding to the training samples;

[0099] The training module 505 is used to train the mini-program recognition model with the goal of minimizing the final loss.

[0100] Optionally, the extraction module 502 is specifically used to: extract features from the image in the training sample through the feature extraction subnetwork to obtain the image features output by the feature extraction subnetwork; extract features from the text in the training sample through the feature extraction subnetwork to obtain the text features output by the feature extraction subnetwork; and fuse the image features and text features to obtain the features of the training sample.

[0101] Optionally, the extraction module 502 is specifically used to: remove at least a portion of the images in the training samples and extract features from the remaining images in the training samples; remove at least a portion of the text in the training samples and extract features from the remaining text in the training samples.

[0102] Optionally, the prediction module 503 is specifically used to: input the training sample into the layout analysis model to obtain the type of each element in the training sample output by the layout analysis model, as a label; and determine the final loss based on the prediction results of the types of each element contained in the page output by the prediction subnet and the label.

[0103] Optionally, the prediction module 503 is specifically used to use the image corresponding to the page as a label, and determine the final loss based on the page restoration result output by the prediction subnet and the label.

[0104] Optionally, the prediction module 503 is specifically used to: determine the service type preset by the service provider of the mini-program for the mini-program, as a label; and determine the final loss based on the prediction result of the service type provided by the mini-program output by the prediction subnet and the label.

[0105] Optionally, the training module 505 is further configured to input the features of the training samples output by the feature extraction subnet into the recognition subnet of the mini-program recognition model, so as to determine the standard type label corresponding to the service provided by the mini-program in the preset standard type labels through the recognition subnet; determine the recognition loss of the mini-program recognition model according to the standard type label output by the recognition subnet and the label of the training samples; and train the mini-program recognition model with the minimization of the recognition loss as the training objective.

[0106] The above describes one or more embodiments of the mini-program recognition method provided in this specification. Based on the same idea, this specification also provides corresponding mini-program recognition devices, such as... Figure 6 As shown.

[0107] Figure 6 This specification provides a schematic diagram of a mini-program recognition device, which specifically includes:

[0108] The input module 601 is used to determine the image corresponding to the page of the mini-program to be identified, which is used as the image to be identified;

[0109] The feature extraction module 602 is used to input the image to be recognized into the feature extraction subnet in the mini-program recognition model to obtain the features of the image to be recognized output by the feature extraction subnet;

[0110] The recognition module 603 is used to input the features into the recognition subnet of the mini-program recognition model and obtain the recognition result output by the recognition subnet, wherein the mini-program recognition model is trained using the model training method described above.

[0111] This specification also provides a computer-readable storage medium storing a computer program that can be used to execute the above-described... Figure 1 Provided model training and Figure 2 The provided method for identifying mini-programs.

[0112] This instruction manual also provides Figure 7 The diagram shows the structure of the electronic device. Figure 7 At the hardware level, the electronic device includes a processor, internal bus, network interface, memory, and non-volatile memory, and may also include other hardware required for the business operations. The processor reads the corresponding computer program from the non-volatile memory into memory and then runs it to achieve the above-mentioned functions. Figure 1 The aforementioned model training and Figure 2 The provided method for identifying mini-programs. Of course, in addition to software implementation, this specification does not exclude other implementation methods, such as logic devices or a combination of hardware and software, etc. In other words, the execution subject of the following processing flow is not limited to individual logic units, but can also be hardware or logic devices.

[0113] In the 1990s, improvements to a technology could be clearly distinguished as either hardware improvements (e.g., improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the methodology). However, with technological advancements, many methodological improvements today can be considered direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved methodology into the hardware circuit. Therefore, it cannot be said that a methodological improvement cannot be implemented using hardware physical modules. For example, a Programmable Logic Device (PLD) (such as a Field Programmable Gate Array (FPGA)) is such an integrated circuit whose logic function is determined by the user programming the device. Designers can program and "integrate" a digital system onto a PLD themselves, without needing chip manufacturers to design and manufacture dedicated integrated circuit chips. Furthermore, nowadays, instead of manually manufacturing integrated circuit chips, this programming is mostly implemented using "logic compiler" software. Similar to the software compiler used in program development, the original code before compilation must be written in a specific programming language, called a Hardware Description Language (HDL). There are many HDLs, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, and RHDL (Ruby Hardware Description Language). Currently, the most commonly used are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should understand that by simply performing some logic programming on the method flow using one of these hardware description languages ​​and programming it into an integrated circuit, the hardware circuit implementing the logical method flow can be easily obtained.

[0114] The controller can be implemented in any suitable manner. For example, it can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicon Labs C8051F320. A memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also recognize that, in addition to implementing the controller in purely computer-readable program code form, the same functionality can be achieved by logically programming the method steps to make the controller take the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the means included therein for implementing various functions can also be considered as structures within the hardware component. Alternatively, the means for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0115] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0116] For ease of description, the above devices are described in terms of function, divided into various units. Of course, in implementing this specification, the functions of each unit can be implemented in one or more software and / or hardware components.

[0117] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0118] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0119] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0120] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0121] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0122] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0123] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0124] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0125] Those skilled in the art will understand that the embodiments of this specification can be provided as methods, systems, or computer program products. Therefore, this specification may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this specification may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0126] This specification can be described in the general context of computer-executable instructions that are executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. This specification can also be practiced in distributed computing environments, where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0127] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0128] The above description is merely an embodiment of this specification and is not intended to limit this specification. Various modifications and variations can be made to this specification by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this specification should be included within the scope of the claims of this application.

Claims

1. A model training method, the method comprising: Identify the images corresponding to the pages of the mini-program and use them as training samples; The training samples are input into the feature extraction subnet of the mini-program recognition model to obtain the features of the training samples output by the feature extraction subnet. At least a portion of the features are input into the prediction subnet of the mini-program recognition model to obtain the prediction results output by the prediction subnet. The prediction results include at least one of the following: the page restoration result, the prediction result of the service type provided by the mini-program, and the prediction result of the type of each element contained in the page. The page restoration result is the information corresponding to the part of the features that is not input into the prediction subnet in the page. The final loss is determined based on the prediction results and the labels corresponding to the training samples; The mini-program recognition model is trained with the goal of minimizing the final loss. The method further includes: After training the mini-program recognition model, the model is fine-tuned. Specifically, the recognition subnet is used to replace the prediction subnet, and the features of the training samples output by the feature extraction subnet are input into the recognition subnet. Through the recognition subnet, the standard type label corresponding to the service provided by the mini-program is determined from the preset standard type labels. Based on the standard type labels output by the recognition subnet and the annotations of the training samples, the recognition loss of the mini-program recognition model is determined; With minimizing the recognition loss as the training objective, the model parameters of the feature extraction subnetwork and the recognition subnetwork are adjusted.

2. The method as described in claim 1, wherein obtaining the features of the training samples output by the feature extraction subnet specifically includes: The feature extraction subnetwork is used to extract features from the images in the training samples to obtain the image features output by the feature extraction subnetwork. The feature extraction subnetwork is used to extract features from the text in the training samples to obtain the text features output by the feature extraction subnetwork. The image features and text features are fused to obtain the features of the training sample.

3. The method as described in claim 2, wherein feature extraction is performed on the images in the training samples, specifically including: At least a portion of the images in the training samples are removed, and features are extracted from the remaining images in the training samples. Feature extraction is performed on the text in the training samples, specifically including: At least a portion of the text in the training samples is removed, and features are extracted from the remaining text in the training samples.

4. The method as described in claim 1, wherein when the prediction result includes prediction results for the types of each element contained in the page, the final loss is determined based on the prediction result and the annotations corresponding to the training samples, specifically including: The training samples are input into the layout analysis model to obtain the type of each element in the training samples output by the layout analysis model, which is then used as a label. The final loss is determined based on the prediction results of the types of each element contained in the page output by the prediction subnet and the annotations.

5. The method as described in claim 1, wherein when the prediction result includes the page restoration result, the final loss is determined based on the prediction result and the annotations corresponding to the training samples, specifically including: The image corresponding to the page is used as a label, and the final loss is determined based on the page reconstruction result output by the prediction subnet and the label.

6. The method as described in claim 1, wherein when the prediction result includes a prediction result of the service type provided by the mini-program, the final loss is determined based on the prediction result and the annotations corresponding to the training samples, specifically including: The service provider for the mini-program is identified and the service type preset for the mini-program is used as a label; The final loss is determined based on the prediction results of the service type provided by the mini-program output by the prediction subnet and the annotation.

7. A method for identifying a mini-program, the method comprising: Identify the image corresponding to the page of the mini-program to be identified, and use it as the image to be identified; The image to be identified is input into the feature extraction subnet of the mini-program recognition model to obtain the features of the image to be identified output by the feature extraction subnet; The features are input into the recognition subnet of the mini-program recognition model to obtain the recognition result output by the recognition subnet, wherein the mini-program recognition model is trained using the method described in any one of claims 1 to 6.

8. An apparatus for model training, the apparatus comprising: The sample extraction module is used to determine the images corresponding to the pages of the mini-program as training samples. The extraction module is used to input the training samples into the feature extraction subnet in the mini-program recognition model to obtain the features of the training samples output by the feature extraction subnet. The prediction module is used to input at least a portion of the features into the prediction subnet of the mini-program recognition model to obtain the prediction result output by the prediction subnet. The prediction result includes at least one of the following: the page restoration result, the prediction result of the service type provided by the mini-program, and the prediction result of the type of each element contained in the page. The page restoration result is the information corresponding to the part of the features that was not input into the prediction subnet in the page. The loss determination module is used to determine the final loss based on the prediction results and the labels corresponding to the training samples; The training module is used to train the mini-program recognition model with the goal of minimizing the final loss. The training module is also used to: fine-tune the model after training the mini-program recognition model, specifically by replacing the prediction subnet with a recognition subnet and inputting the features of the training samples output by the feature extraction subnet into the recognition subnet, so as to determine the standard type label corresponding to the service provided by the mini-program in the preset standard type labels through the recognition subnet; Based on the standard type labels output by the recognition subnet and the annotations of the training samples, the recognition loss of the mini-program recognition model is determined; With minimizing the recognition loss as the training objective, the model parameters of the feature extraction subnetwork and the recognition subnetwork are adjusted.

9. A device for identifying mini-programs, the device comprising: The input module is used to determine the image corresponding to the page of the mini-program to be identified, which is then used as the image to be identified. The feature extraction module is used to input the image to be recognized into the feature extraction subnet in the mini-program recognition model to obtain the features of the image to be recognized output by the feature extraction subnet; The recognition module is used to input the features into the recognition subnet of the mini-program recognition model and obtain the recognition result output by the recognition subnet, wherein the mini-program recognition model is trained using the method described in any one of claims 1 to 6.

10. A computer-readable storage medium storing a computer program that, when executed, implements the method of any one of claims 1 to 7.

11. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method of any one of claims 1 to 7.

Images