Interface generation method and device, electronic equipment and storage medium

Abstract

The invention discloses an interface generation method and device, electronic equipment and a storage medium, and relates to the field of image processing. The interface generation method comprises the steps of determining a to-be-recognized image feature matrix corresponding to a to-be-recognized component image included in a to-be-recognized image; inputting the to-be-recognized image feature matrix into a recognition model to obtain an identifier corresponding to the to-be-recognized component; generating an interface configuration file based on the identifier corresponding to the to-be-identified component; and based on the interface configuration file, generating an interface comprising the to-be-identified component.

Description

Technical Field

[0001] This application belongs to the field of image processing, and particularly relates to an interface generation method, apparatus, electronic device, and storage medium. Background Art

[0002] In the software development process of equipment, front-end engineers need to draw corresponding software interfaces for water / gas / electricity, etc. according to complex mechanical drawings. The process of manually creating a software interface includes development steps such as developers drawing elements, layout, element rendering, adding software functions, and integration testing according to mechanical drawings. The production process usually consumes a large amount of time and energy of developers, prolongs the development cycle, increases the development cost, and there is also a risk of accidents due to errors in manual production. Summary of the Invention

[0003] The purpose of the embodiments of this application is to provide an interface generation method, apparatus, electronic device, and storage medium, which can solve the problems of low efficiency and high error rate in manually creating software interfaces, and improve the efficiency and accuracy of interface production.

[0004] To solve the above technical problems, this application is implemented as follows:

[0005] In a first aspect, an embodiment of this application provides an interface generation method, which includes: determining a to-be-recognized image feature matrix corresponding to a to-be-recognized component image included in a to-be-recognized image; inputting the to-be-recognized image feature matrix into a recognition model to obtain an identifier corresponding to the to-be-recognized component; generating an interface configuration file based on the identifier corresponding to the to-be-recognized component; and generating an interface including the to-be-recognized component based on the interface configuration file.

[0006] In a second aspect, an embodiment of this application provides an electronic device, which includes a processor and a memory. The memory stores a program or instruction that can run on the processor, and when the program or instruction is executed by the processor, the steps of the interface generation method described in the first aspect are implemented.

[0007] In a third aspect, an embodiment of this application provides a readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed by a processor, the steps of the interface generation method described in the first aspect are implemented.

[0008] In an embodiment of the present application, by determining a to-be-recognized image feature matrix corresponding to a to-be-recognized component image included in a to-be-recognized image; inputting the to-be-recognized image feature matrix into a recognition model to obtain an identifier corresponding to the to-be-recognized component; generating an interface configuration file based on the identifier corresponding to the to-be-recognized component; and generating an interface including the to-be-recognized component based on the interface configuration file, the automation of interface production is realized, and the components in the drawing are recognized by the recognition model, which solves the problem of inevitable misoperations in the manual production process, improves the recognition accuracy, also avoids the problem of low efficiency in manually producing a software interface, and improves the efficiency and accuracy of interface production. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments recorded in the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0010] Figure 1 It is a schematic flowchart of a method for generating an interface provided by an embodiment of the present application.

[0011] Figures 2 - 3 It is an example diagram of a method for generating an interface provided by an embodiment of the present application.

[0012] Figure 4 It is a schematic flowchart of a training method of a recognition model provided by an embodiment of the present application.

[0013] Figure 5a It is an example diagram of a to-be-recognized image including a component image provided by an embodiment of the present application.

[0014] Figures 5b - 5e It is an example effect diagram of preprocessing a to-be-recognized image provided by an embodiment of the present application.

[0015] Figure 5f It is an example effect diagram of an equivalent image feature matrix provided by an embodiment of the present application.

[0016] Figure 6 It is a schematic hardware structure diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0017] To enable those skilled in the art to better understand the technical solutions in this application, the following will clearly and completely describe the technical solutions in the embodiments of this application in conjunction with the accompanying drawings in the embodiments of this application. Obviously, the described embodiments are only a part of the embodiments of this application, rather than all the embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of this application.

[0018] Figure 1 FIG. 4 shows a schematic flowchart of an interface generation method provided by an embodiment of this application. This method can be executed by an electronic device. In other words, this method can be executed by software or hardware installed in this electronic device. As Figure 1 shown, this method may include the following steps:

[0019] S101: Determine a to-be-recognized image feature matrix corresponding to the to-be-recognized component image included in the to-be-recognized image.

[0020] Obtain the to-be-recognized image, where the to-be-recognized image includes images of at least one to-be-recognized component. The to-be-recognized image can be a mechanical drawing and / or a circuit diagram, etc. Exemplarily, the component can be a bearing or a spring in a mechanical drawing, or a resistor or a capacitor in an electrical diagram. For example, if the to-be-recognized image is a circuit diagram, and the circuit diagram includes images of a resistor and a capacitor, the images of the resistor and the capacitor can be used as the to-be-recognized component images.

[0021] Determine the to-be-recognized image feature matrix corresponding to the to-be-recognized component image. The to-be-recognized image feature matrix can be a pixel matrix, where the pixel matrix can be a binary matrix consisting of only 0 and 1, and the 1 in the pixel matrix can represent the contour of the to-be-recognized component image. Combining Figure 5a 、 Figure 5b shown, the to-be-recognized component image can be as Figure 5a shown, and the to-be-recognized image feature matrix corresponding to the to-be-recognized component image can be as Figure 5b shown, Figure 5b The 1 in Figure 5a represents the contour of the to-be-recognized component image.

[0022] In the case where the to-be-recognized image includes images of multiple components, the to-be-recognized image feature matrix can represent the images of all the components included in the to-be-recognized image, or can represent the images of some of the components included in the to-be-recognized image. In the case where the to-be-recognized image includes images of multiple components and the to-be-recognized image feature matrix represents the images of some of the components included in the to-be-recognized image, this step can determine multiple to-be-recognized image feature matrices.

[0023] S102: Input the feature matrix of the image to be recognized into the recognition model to obtain the identifier corresponding to the component to be recognized.

[0024] Specifically, in this step, the feature matrix of the image to be recognized is input into the recognition model, and the recognition model recognizes the image to be recognized. The recognition result is the identifier corresponding to the component to be recognized, and this identifier is used to represent at least one component included in the image to be recognized. For example, identifier 1 represents a resistor, and identifier a represents a spring, etc.

[0025] In one implementation, the identifier can also be used to represent the position of the component to be recognized. When the image to be recognized is a circuit diagram, a complex circuit diagram can be divided into multiple simple circuit structures, and the position of the component in the circuit diagram can be marked by the identifier. For example, a simple circuit structure can include a power supply and a load, then the identifier can be used to mark whether the component to be recognized is the power supply or the load in the circuit. A complex circuit can include multiple branches, and each branch includes a power supply and a load. The identifier can be used to mark which branch's power supply or load the component to be recognized is. The embodiments of the present application do not limit the corresponding relationship between the identifier and the component to be recognized, nor the corresponding relationship between the identifier and the position.

[0026] S103: Generate an interface configuration file based on the identifier corresponding to the component to be recognized.

[0027] The interface configuration file contains the information required to successfully generate an interface. In this step, an interface configuration file is generated based on the identifier corresponding to the component to be recognized. When the identifier is used to represent the corresponding component to be recognized, an interface can be generated by running this interface configuration file, and the image of the component to be recognized is displayed on this interface. When the identifier is also used to represent the position of the component to be recognized, running this interface configuration file can generate an interface, and the component to be recognized is displayed at the position corresponding to the identifier.

[0028] Optionally, the format of the above interface configuration file can be one of JavaScript Object Notation (JSON), YAML (YAML Ain't Markup Language, a format for expressing data serialization), and Extensible Markup Language (XML).

[0029] S104: Generate an interface including the component to be recognized based on the interface configuration file.

[0030] In this step, an interface including the components is generated by reading and running an interface configuration file. It should be noted that the above interface can be a control interface or a display interface in an application, or a screen display interface of an electronic device with a screen, which is not limited herein.

[0031] A method for generating an interface provided by an embodiment of the present application includes determining a to-be-recognized image feature matrix corresponding to a to-be-recognized component image included in a to-be-recognized image; inputting the to-be-recognized image feature matrix into a recognition model to obtain an identifier corresponding to the to-be-recognized component; generating an interface configuration file based on the identifier corresponding to the to-be-recognized component; and generating an interface including the to-be-recognized component based on the interface configuration file, realizing the automation of interface production. Moreover, the components in the drawing are recognized by the recognition model, which solves the problem of inevitable misoperations in the manual production process, improves the recognition accuracy, and also avoids the problem of low efficiency in manually producing a software interface, thereby improving the efficiency and accuracy of interface production.

[0032] In one implementation, step S103 may include the following steps:

[0033] Determine information of an interface control associated with the identifier corresponding to the to-be-recognized component. The information of the interface control may include attribute information and layout information of the control. The attribute information includes name, size, etc., and the layout information includes connection relationship, position information, etc. Specifically, before this step, the image information represented by the identifier corresponding to the component may be associated with the information of the control. For example, the name of the component may be associated with the name of the control. For example, the identifier a corresponding to a spring is associated with the connection control A in the interface. Or the connection relationship of the component may be associated with the layout information of the control. For example, if a spring is fixed on a connecting rod, the connection relationship of the connection control A in the corresponding interface is fixed above the connection control B, and the connection relationship of the connection control B is fixed below the connection control A.

[0034] Figures 2 - 3 This is an example diagram of the interface generation method provided by an embodiment of the present application. As Figure 2 shown, optionally, the above interface may be a Winform interface, and the association between the identifier corresponding to the above component and the information of the control of the Winform interface is realized through the automatic control tool class DynamicControls.cs. Figure 3Shows the database (DB) data and function body of the DynamicControls.cs control. For example, when the function body of the "On / Off Animation" function is opened, an interface configuration file is generated. When generating the interface including the components based on this interface configuration file in this step, the animation effect of generating this interface can be displayed. For example, if a spring is fixed to a connecting rod, the interface can first display the spring through animation, then display the connecting rod through animation, and then display the spring fixed to the connecting rod through animation. When the function body of the "On / Off Animation" function is closed, an interface configuration file is generated. When generating the interface including the components based on this interface configuration file in this step, the state where the spring is fixed to the connecting rod can be displayed without showing the above animation content.

[0035] Determine an interface generation command based on the information of the interface control, where there is an association between the information of the interface control and the information of the interface control. Exemplarily, the association between the component and the attribute information of the control, determining the command to create the control in the interface based on the attribute information of the control; or the association between the relative coordinates and the position information of the control, determining the command to set the display position of the control in the interface based on the position information of the control.

[0036] Generate an interface configuration file based on the interface generation command. In this step, based on the above interface generation command, an interface configuration file is generated to obtain the information required for successfully generating the interface.

[0037] In one implementation, the interface further includes events of the control. To obtain a dynamic interface, after step S104, the method further includes:

[0038] Obtain the text content corresponding to the image to be recognized. The text content is used to indicate the actions performed by the components. Multiple such text contents can be preset before this step, and each text content is used to indicate the actions performed by at least one of the components. After obtaining the image to be recognized, the identifier of at least one component included in the image to be recognized can be determined, and the text content corresponding to the image to be recognized can be determined from the multiple text contents according to the identifier of the component, and the actions performed by the component can be determined according to the text content, such as the opening and closing of a valve.

[0039] Furthermore, the text content can be associated with the events of the control, so that the control performs the actions executed by the component in the interface. Specifically, the actions indicated by the text content are associated with the events of the control, such as being associated with changing the opening and closing state of the control, so that the control performs the actions executed by the component in the interface, such as opening and closing the control, thereby dynamically presenting the content of the image to be recognized in the interface, and more vividly and intuitively representing the image to be recognized. Optionally, according to the association between the text content and the events of the control, an interface generation command is generated and written into the interface configuration file; or the events of the control are manually set in the interface.

[0040] In one implementation, in the case of multiple component images to be recognized in the image to be recognized, S101 may include: segmenting the image to be recognized to obtain multiple segmented images to be recognized, and one segmented image to be recognized includes one component image to be recognized. For example, the image to be recognized is a circuit diagram of a power supply connected to a light bulb, which includes 2 component images to be recognized, one is the power supply and the other is the light bulb. The image to be recognized can be segmented to obtain 2 segmented images to be recognized, one segmented image to be recognized includes the power supply, and the other includes the light bulb. Determine the respective image feature matrices to be recognized corresponding to the respective segmented images to be recognized. For example, determine the image feature matrices to be recognized corresponding to the above 2 segmented images to be recognized, that is, the power supply corresponds to matrix 1 and the light bulb corresponds to matrix 2.

[0041] Thus, the embodiment of the present application can avoid recognition errors caused by the overlap between components in the image to be recognized in the case where the image to be recognized includes multiple components, and improve the accuracy rate when matching and recognizing components in subsequent steps.

[0042] In one implementation, before S102, the method further includes: preprocessing the image feature matrix to be recognized; the preprocessing includes at least one of the following:

[0043] Dilation;

[0044] Erosion;

[0045] Noise removal;

[0046] Filtering.

[0047] Specifically, as shown in combination with Figures 5a - 5e Load the component image to be recognized in the image to be recognized shown in Figure 5a After loading is completed, the image feature matrix to be recognized corresponding to the component image to be recognized is obtained, with reference to Figure 5bAs shown, the shape represented by the image feature matrix to be recognized is different from the image of the component to be recognized. In this case, a fixed-length matrix can be used as a template to filter out the data noise in the image feature matrix to be recognized and reduce the error in the pixel matrix. First, perform dilation processing on the above pixel matrix, and the result of the dilation processing is referred to Figure 5c As shown, the noise range in the image feature matrix to be recognized expands. Perform erosion processing, and the result of the erosion processing is referred to Figure 5d As shown, the shape represented by the pixel matrix after the erosion processing is relatively close to the shape of the component. Use Gaussian filtering to Figure 5d Perform smoothing processing on the pixel matrix shown, and the result of the smoothing processing is referred to Figure 5e As shown, the shape represented by the obtained pixel matrix is basically the same as the shape of the image of the component to be recognized, and more accurately reflects the characteristics of the image of the component to be recognized.

[0048] Figure 4 The flowchart of a training method of an identification model is shown. In order to improve the accuracy of the identification model in identifying components, before inputting the image to be recognized into the identification model, the identification model needs to be trained. As Figure 4 shown, the training includes the following steps:

[0049] S201: Determine the effective sample set.

[0050] Obtain multiple alternative training component images. The alternative training component images can be from alternative training images, where the alternative training images include multiple alternative training component images. According to the alternative training component images, determine the alternative training sample matrix corresponding to the alternative training component images. In one implementation, the alternative training component images can be binarized to obtain the alternative training sample matrix.

[0051] In one implementation, a Gaussian filtering method can be used to filter the alternative training sample matrix to generate a differentiated effective sample set. Specifically, the Gaussian filtering method includes: primary filtering and secondary filtering During the iteration process, the number of filtering times and the filtering order can be changed. For example, two primary filterings can be used first and then secondary filtering, or one primary filtering and two secondary filterings can be used, etc., thereby generating a differentiated effective sample set.

[0052] In one implementation, when the similarity between multiple alternative training sample matrices meets a predetermined similarity requirement, these multiple alternative training sample matrices are determined as training sample matrices. For example, if an alternative training image includes resistor 1, resistor 2, capacitor 3, and inductor 4, then multiple alternative training component images are obtained, including the images of resistor 1, resistor 2, capacitor 3, and inductor 4. Based on these images, the matrices corresponding to resistor 1, resistor 2, capacitor 3, and inductor 4 are determined as alternative training sample matrices. Among them, when the similarity between the matrices corresponding to resistor 1 and resistor 2 meets the predetermined similarity requirement, the matrices corresponding to resistor 1 and resistor 2 are determined as training sample matrices and added to the effective sample set. The matrices corresponding to capacitor 3 and inductor 4 do not meet the predetermined similarity requirement and do not belong to the training sample matrices. Thus, the effective sample set includes the training sample matrices corresponding to multiple training component images, and the similarity between multiple said training sample matrices meets the predetermined similarity requirement.

[0053] To expand the effective sample set, in one implementation, this step further includes: determining an equivalent image feature matrix equivalent to the training sample matrix; adding the equivalent image feature matrix to the effective sample set. Among them, the equivalent image feature matrix can obtain the training sample matrix through a finite number of transformations, and the components corresponding to the equivalent image feature matrix and the training sample matrix are actually the same. Thus, the components corresponding to the equivalent image feature matrix can also be recognized by the recognition model, further improving the recognition rate of the components to be recognized.

[0054] In one implementation, v n (k) can represent the training sample matrix, and v n-1 (k) can represent the equivalent image feature matrix equivalent to the training sample matrix. The training sample matrix and its equivalent matrix satisfy v n-1 (k) = G(Δt, k)v n (k). Combining Figure 5f , it can be understood that v n-1 (k) can be used to represent the matrix corresponding to ① in the figure, and v n-1 (k) can be used to represent the matrix corresponding to ② or ③ in the figure. ①, ②, and ③ are the same type of component but different in size, corresponding to different matrices, and these matrices are equivalent matrices to each other. This step can obtain different sample matrices of the same component.

[0055] The above G(Δt, k) is an amplification matrix. Since the exact solution and the numerical solution of the linear constant coefficient difference equation both satisfy the same equation, the error of the solution satisfies the same homogeneous linear equation. For this equation, the Fourier stability analysis method can be used to analyze whether the error increases or decays over time to determine the stability of the difference equation. The amplification matrix is defined as the ratio of the Fourier transform of the solutions on two adjacent time levels of the homogeneous linear equation:

[0056]

[0057] where u n (x) is a p-dimensional vector that satisfies the constant coefficient linear difference equation: u n-1 (x) = C(Δt)u n (x), and v n (k) represents the Fourier coefficient of u n (x) with a wave number of k.

[0058] This step can determine the equivalent image feature matrix equivalent to the training sample matrix through various mathematical algorithms, and the embodiments of the present application do not make specific limitations on this.

[0059] Optionally, the similarity between the training sample matrix and the equivalent image feature matrix can meet the predetermined similarity requirement. In the case where there are multiple equivalent image feature matrices, the similarity between the multiple equivalent image feature matrices can meet the predetermined similarity requirement. For example, it is higher than the similarity threshold, etc.

[0060] In one implementation, determining the equivalent image feature matrix equivalent to the training sample matrix corresponding to the training component image may include: determining the alternative image feature matrix corresponding to the alternative component image included in the alternative sample image; and determining the equivalent image feature matrix equivalent to the training sample matrix from the alternative image feature matrices. For example, the alternative component images included in the alternative sample image include resistor 2, capacitor 3, and inductor 4, and their corresponding alternative image feature matrices are determined respectively; from the alternative image feature matrices, if the matrix corresponding to resistor 2 is equivalent to the training sample matrix (for example, the matrix corresponding to resistor 1), then the matrix corresponding to resistor 2 is determined as the equivalent image feature matrix. If the matrices corresponding to capacitor 3 and inductor 4 have an equivalent relationship with the training sample matrix (for example, the matrix corresponding to resistor 1), they cannot be determined as the equivalent image feature matrix.

[0061] S202: Input the training sample matrix in the effective sample set into the recognition model to obtain the identifier corresponding to the training component image.

[0062] This identifier is used to characterize the training components. For example, identifier 1 represents a resistor, and identifier a represents a spring, etc. In one implementation, the identifier can also be used to characterize the position of the training components. In the case where the alternative training image is a circuit diagram, the structure of the circuit diagram can be analyzed, the complex circuit diagram can be divided into multiple simple circuit structures, and the position of the components in the circuit diagram can be marked by identifiers. The simple circuit structure can include: a power supply and a load, then the identifier can be used to mark whether the training component is the power supply or the load in the circuit. The complex circuit can include multiple branches, each branch includes a power supply and a load, and the identifier can be used to mark which branch's power supply or load the training component is.

[0063] S203: Adjust the model parameters of the recognition model based on the identifier corresponding to the training component until the convergence condition is met.

[0064] Adjust the model parameters of the recognition model based on the identifier corresponding to the training component. When the output of the recognition model meets the convergence condition, it is determined that the training of the recognition model is completed. Thus, the accuracy of model training is improved.

[0065] In one implementation, before inputting the training sample matrix in the effective sample set into the recognition model, the method further includes: preprocessing the training sample image; the preprocessing includes at least one of the following:

[0066] Dilation;

[0067] Erosion;

[0068] Noise removal;

[0069] Filtering.

[0070] Specifically, as shown in combination with Figures 5a - 5e Load the training component image shown in Figure 5a After loading, the training sample matrix corresponding to the training component image is obtained. Referring to Figure 5b shown, the shape represented by the training sample matrix is different from that of the training component image. In this case, a fixed-length matrix is used as a template to filter the data noise in the training sample matrix and reduce the error in the pixel matrix. First, the above pixel matrix is subjected to dilation processing, and the processing result is shown in Figure 5c shown, and the noise range in the training sample matrix expands. Then, erosion processing is performed, and the processing result is shown in Figure 5d shown. After erosion processing, the shape represented by the pixel matrix is closer to the shape of the component. Gaussian filtering is used to smooth the pixel matrix shown in Figure 5d shown, and the processing result is shown in Figure 5eAs shown, the shape represented by the obtained pixel matrix is basically the same as the shape of the training component image, and more accurately reflects the features of the training component image.

[0071] An interface generation method provided by an embodiment of the present application can make the training sample matrix closer to the physical component through preprocessing, thereby improving the accuracy of identifying the image to be recognized.

[0072] Figure 6 The figure shows a schematic hardware structure diagram of an electronic device provided by an embodiment of the present application. Referring to this figure, at the hardware level, the electronic device includes a processor, and optionally, an internal bus, a network interface, and a memory. Among them, the memory may include a memory, such as a high-speed random access memory (Random-Access Memory, RAM), and may also include a non-volatile memory, such as at least one disk memory, etc. Of course, the electronic device may also include other hardware required for other services.

[0073] The processor, the network interface, and the memory can be interconnected through an internal bus. The internal bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For the sake of simplicity, only a bidirectional arrow is used to represent it in this figure, but it does not mean that there is only one bus or one type of bus.

[0074] The memory is used to store programs. Specifically, the program may include program code, and the program code includes computer operation instructions. The memory may include a memory and a non-volatile memory, and provide instructions and data to the processor.

[0075] The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it, and forms a device for positioning the target user at the logical level. The processor executes the program stored in the memory and is specifically used to execute: each step of the interface generation method described in at least one of the above method embodiments.

[0076] The methods disclosed in the embodiments shown in the flowcharts of the present application as described above can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with the ability to process signals. During implementation, the steps of the above methods can be completed by the integrated logic circuit in the hardware of the processor or instructions in software form. The above-mentioned processor may be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. It can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The steps of the method disclosed in combination with the embodiments of the present application can be directly embodied as being executed and completed by a hardware decoding processor, or executed and completed by a combination of hardware and software modules in the decoding processor. The software module may be located in a mature storage medium in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. This storage medium is located in the memory, and the processor reads the information in the memory and combines its hardware to complete the steps of the above method.

[0077] The electronic device can also execute the interface generation method described in at least one of the foregoing method embodiments and can achieve the same technical effects as the foregoing method embodiments, which will not be elaborated here.

[0078] Of course, in addition to the software implementation, the electronic device of the present application does not exclude other implementation manners, such as a logic device or a combination of software and hardware, etc. That is to say, the execution subject of the following processing flow is not limited to each logic unit and may also be hardware or a logic device.

[0079] The embodiments of the present application also propose a computer-readable storage medium. The computer-readable medium stores one or more programs. When the one or more programs are executed by an electronic device including a plurality of application programs, the electronic device is caused to execute the steps of the interface generation method described in at least one of the foregoing method embodiments.

[0080] Among them, the computer-readable storage medium includes a read-only memory (ROM), a random access memory (RAM), a magnetic disk, an optical disc, etc.

[0081] Furthermore, an embodiment of the present application also provides a computer program product. The computer program product includes a computer program stored on a non-transitory computer-readable storage medium. The computer program includes program instructions. When the program instructions are executed by a computer, the steps of at least one of the above method embodiments are implemented.

[0082] In summary, the above are only the preferred embodiments of the present application and are not used to limit the protection scope of the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

[0083] The systems, devices, modules or units illustrated in the above embodiments can be specifically implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer can be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or any combination of these devices.

[0084] Computer-readable media includes both permanent and non-permanent, removable and non-removable media and can be implemented by any method or technology for storing information. The information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media such as modulated data signals and carrier waves.

[0085] It should also be noted that the term "comprise", "include" or any other variant thereof is intended to cover non-exclusive inclusion, such that a process, method, commodity or device comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, commodity or device. Without further limitation, an element defined by the statement "comprising an..." does not exclude the presence of additional identical elements in the process, method, commodity or device comprising said element.

[0086] Each embodiment in this specification is described in a progressive manner. For the identical and similar parts among the embodiments, reference can be made to each other. Each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and reference can be made to the relevant part of the method embodiment for the related content.

Claims

1. An interface generation method, characterized in that, it includes: Determine a to-be-recognized image feature matrix corresponding to a to-be-recognized component image included in the to-be-recognized image; Input the to-be-recognized image feature matrix into a recognition model to obtain an identifier corresponding to the to-be-recognized component; Generate an interface configuration file based on the identifier corresponding to the to-be-recognized component; Generate an interface including the to-be-recognized component based on the interface configuration file.

2. The interface generation method according to claim 1, characterized in that, before inputting the to-be-recognized image feature matrix into the recognition model, it further includes: Determine a valid sample set; the valid sample set includes training sample matrices corresponding to multiple training component images, and the similarity between multiple said training sample matrices meets a predetermined similarity requirement; Input the training sample matrices in the valid sample set into the recognition model to obtain identifiers corresponding to the training components; Adjust the model parameters of the recognition model based on the identifiers corresponding to the training components until a convergence condition is met.

3. The interface generation method according to claim 2, characterized in that, after determining the valid sample set, it further includes: Determine an equivalent image feature matrix equivalent to the training sample matrix; Add the equivalent image feature matrix to the valid sample set.

4. The interface generation method according to claim 3, characterized in that, the determining of the equivalent image feature matrix equivalent to the training sample matrix corresponding to the training component image includes: Determine an alternative image feature matrix corresponding to an alternative component image included in the alternative sample image; Determine an equivalent image feature matrix equivalent to the training sample matrix from the alternative image feature matrix.

5. The interface generation method according to claim 2, characterized in that, before inputting the training sample matrices in the valid sample set into the recognition model, it further includes: Preprocess the training sample matrix; The preprocessing includes at least one of the following: Dilation; Erosion; Noise elimination; Filtering.

6. The interface generation method according to claim 1, characterized in that, before inputting the to-be-recognized image feature matrix into the recognition model, the recognition model further includes: Preprocess the to-be-recognized image feature matrix; Dilation; Erosion; Noise elimination; Filtering.

7. The interface generation method according to claim 1, characterized in that, in the case of multiple to-be-recognized component images in the to-be-recognized image, the determining of the to-be-recognized image feature matrix corresponding to the to-be-recognized component image included in the to-be-recognized image includes: Segment the to-be-recognized image to obtain multiple to-be-recognized segmented images, and one said to-be-recognized segmented image includes one to-be-recognized component image; Determine the to-be-recognized image feature matrices corresponding to the respective to-be-recognized segmented images.

8. The interface generation method according to claim 1, characterized in that, the generating of the interface configuration file based on the identifier corresponding to the to-be-recognized component includes: Determine information of an interface control associated with the identifier corresponding to the to-be-recognized component; Determine an interface generation command based on the information of the interface control; Generate an interface configuration file based on the interface generation command.

9. The interface generation method according to claim 8, wherein, the interface further includes events of the control; after generating the interface including the component, further includes: obtain the text content corresponding to the image to be recognized, where the text content is used to indicate the action performed by the component; associate the text content with the event of the control, so that the control performs the action performed by the component in the interface.

10. An electronic device, wherein, it includes a processor, a memory, and a program or instruction stored on the memory and executable on the processor. When the program or instruction is executed by the processor, the steps of the interface generation method according to any one of claims 1-9 are implemented.

11. A readable storage medium, wherein, a program or instruction is stored on the readable storage medium, and when the program or instruction is executed by a processor, the steps of the interface generation method according to any one of claims 1-9 are implemented.

Images