Page data generation method and apparatus, and device and storage medium

Abstract

Disclosed in the present application are a page data generation method and apparatus, and a device and a storage medium. The method comprises: acquiring a user interface design image comprising N image segments; on the basis of the N image segments, determining components corresponding to the N image segments; generating layout information of the components according to position information, in the user interface design image, of target image segments corresponding to the components; and generating page development data on the basis of the components and the layout information of the components, wherein the page development data is used for generating a user interface corresponding to the user interface design image.

Description

Page data generation method, device, equipment and storage medium

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to Chinese patent application 202311134907.2, filed on September 4, 2023, entitled “Page data generation method, device, equipment and storage medium,” and the entire contents of that application are incorporated herein by reference. Technical Field

[0003] The present application relates to the field of computer technology, and in particular to a method, apparatus, device, and storage medium for generating page data. Background Art

[0004] During the page development process for mobile applications and internet web pages, developers can analyze the required page components and their layout based on the user interface (UI) design draft. Then, developers can manually edit the page components and their layout to complete page development. However, this process requires manual editing, consumes a significant amount of page development time, and reduces page development efficiency.

[0005] Summary of the Invention

[0006] The embodiments of the present application provide a page data generation method, apparatus, device and storage medium, which can solve the problem of manually editing page development data in related technologies, wasting page development time and reducing page development efficiency.

[0007] In a first aspect, an embodiment of the present application provides a method for generating page data, which may include:

[0008] Acquire a user interface design image, where the user interface design image includes N image segments, where N is a positive integer;

[0009] Based on the N image segments, determining components corresponding to the N image segments;

[0010] Generate layout information of the component according to position information of the target image segment corresponding to the component in the user interface design image;

[0011] Page development data is generated based on the components and the layout information of the components. The page development data is used to generate a user interface corresponding to the user interface design image.

[0012] In a second aspect, an embodiment of the present application provides a device for generating page data, which may include:

[0013] An acquisition module is used to acquire a user interface design image, where the user interface design image includes N image segments, where N is a positive integer;

[0014] a determination module, configured to determine components corresponding to the N image segments based on the N image segments;

[0015] A generation module, configured to generate layout information of the component according to position information of the target image segment corresponding to the component in the user interface design image;

[0016] The generation module is further configured to generate page development data based on the components and the layout information of the components, and the page development data is used to generate a user interface corresponding to the user interface design image.

[0017] In a third aspect, an embodiment of the present application provides a computer device, the computer device comprising: a processor and a memory storing computer program instructions;

[0018] When the processor executes the computer program instructions, the page data generating method shown in the first aspect is implemented.

[0019] In a fourth aspect, an embodiment of the present application provides a computer storage medium having computer program instructions stored thereon. When the computer program instructions are executed by a processor, the page data generation method shown in the first aspect is implemented.

[0020] In a fifth aspect, an embodiment of the present application provides a chip, which includes a processor and a communication interface, the communication interface and the processor are coupled, and the processor is used to run programs or instructions to implement the page data generation method shown in the first aspect.

[0021] In a sixth aspect, an embodiment of the present application provides a computer program product, which is stored in a storage medium and is executed by at least one processor to implement the page data generation method shown in the first aspect.

[0022] The page data generation method, apparatus, device, and storage medium of the embodiments of the present application obtain a user interface design image, which includes N image segments; based on the N image segments, determine the components corresponding to the N image segments; generate component layout information according to the position information of the target image segments corresponding to the components in the user interface design image; and generate page development data based on the components and the component layout information, the page development data being used to generate a user interface corresponding to the user interface design image. In this way, by analyzing the user interface design image, information about each component and the component layout information in the user interface design image is obtained, and based on the components and the component layout information, page development data for generating a user interface corresponding to the user interface design image is generated. This eliminates the need for users to manually edit page code, reduces user participation, shortens page development time, and improves page development efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0024] FIG1 is a flow chart of a method for generating page data provided by an embodiment of the present application;

[0025] FIG2 is a schematic diagram of the structure of a page data generating device provided in an embodiment of the present application;

[0026] FIG3 is a schematic diagram of the structure of a computer device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0027] The features and exemplary embodiments of various aspects of the present application will be described in detail below. In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain the present application, rather than to limit the present application. For those skilled in the art, the present application can be implemented without the need for some of these specific details. The following description of the embodiments is merely to provide a better understanding of the present application by illustrating the examples of the present application.

[0028] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or device comprising the element.

[0029] The acquisition, storage, use, and processing of data (including but not limited to the features and information herein) in the technical solution of this application comply with the relevant provisions of national laws and regulations.

[0030] In the related art, during the page development process for mobile applications and internet web pages, developers can analyze the required page components and their layout based on the user interface (UI) design draft. Then, developers can manually edit the page components and their layout to complete page development. However, due to the repetitive nature of page component coding, the extensive and repetitive component analysis and layout design consumes a significant amount of page development time, making it impossible to quickly generate page code and reducing page development efficiency.

[0031] In order to solve the problems in the related art, the embodiments of the present application provide a page data generation method, device, computer equipment and storage medium. To this end, the embodiments of the present application, with the help of the development of image processing technology and artificial intelligence technology, propose a page data generation method based on user interface design images, so as to obtain information about each component in the user interface design image and the layout information of the components by analyzing the user interface design image, and generate page development data for generating a user interface corresponding to the user interface design image based on the components and the layout information of the components, without the need for users to manually edit the page code, thereby reducing user participation, shortening page development time, and improving page development efficiency.

[0032] The following will describe in detail the page data generation method, apparatus, computer device and storage medium of the embodiments of the present application with reference to Figures 1 to 3. It should be noted that these embodiments are not intended to limit the scope of the disclosure of the present application.

[0033] First, a page data generation method provided in an embodiment of the present application is described with reference to FIG1 , as shown below.

[0034] FIG1 is a flow chart of a method for generating page data provided in an embodiment of the present application.

[0035] As shown in FIG1 , the page data generation method may be applied to a computer device, and the page data generation method may specifically include the following steps:

[0036] Step 110, obtaining a user interface design image, where the user interface design image includes N image segments, where N is a positive integer; Step 120, determining components corresponding to the N image segments based on the N image segments; Step 130, generating layout information of the components according to position information of target image segments corresponding to the components in the user interface design image; Step 140, generating page development data based on the components and their layout information, where the page development data is used to generate a user interface corresponding to the user interface design image.

[0037] In this way, by analyzing the user interface design image, the information of each component in the user interface design image and the layout information of the components are obtained, and based on the components and the layout information of the components, page development data for generating a user interface corresponding to the user interface design image is generated. There is no need for users to manually edit the page code, which reduces user participation, shortens page development time, and improves page development efficiency.

[0038] The above steps are described in detail below.

[0039] First, referring to step 110, in one or more possible embodiments, before step 110, the page data generation method may further include:

[0040] Step 150: perform image preprocessing on the user interface design image to obtain N image segments.

[0041] Furthermore, the above-mentioned step 150 may specifically include:

[0042] Step 1501: binarize the user interface design image to obtain a binarized image.

[0043] Step 1502: Enhance the binary image using a histogram equalization algorithm to obtain a histogram equalized image.

[0044] Step 1503: performing dilation and erosion processing on the histogram equalization image to obtain a dilated and eroded image;

[0045] Step 1504: Process the dilated and eroded images using an edge detection algorithm to obtain an edge detection image, where the edge detection image includes an edge region.

[0046] Step 1505 , locating the closed area in the edge detection image by flood filling and edge area;

[0047] Step 1506 : Segment the user interface design image into N image segments according to the closed areas.

[0048] Exemplarily, the user interface design image is subjected to image preprocessing, that is, the user interface design image is first binarized to obtain a binarized image; then, the edge area in the binarized image is enhanced by a histogram equalization algorithm to obtain a histogram equalized image; further, in order to make the edge area in the image clearer, the histogram equalized image can be dilated and eroded to obtain a dilated and eroded image, and the edge area in the dilated and eroded image is clearer; then, the dilated and eroded image is processed by an edge detection algorithm to obtain an edge detection image including the edge area, and these edge areas are flood-filled and mask differences are calculated to obtain a series of closed areas in the user interface design image, and these closed areas are segmented to obtain the pre-segmentation processing result of the user interface design image, that is, the pre-segmentation processing result can be N image fragments in the user interface design image.

[0049] Next, regarding step 120, the components corresponding to the N image segments in the embodiment of the present application may include at least one of the following:

[0050] A first component matching a first image segment among the N image segments, a second component matching a first sub-image segment of a second image segment among the N image segments, and a third component preset for a second sub-image segment of the second image segment.

[0051] Based on the above different types of image segments, the process of determining components is given below with different examples, as shown below.

[0052] In one or more possible embodiments, the components corresponding to the N image segments include a first component matching the first image segment. Based on this, step 120 may specifically include:

[0053] Step 1201: Perform image recognition on N image segments using a component recognition model to obtain a first recognition result.

[0054] Step 1202: When the first identification result includes a first image segment that can match a component among the N image segments, a first component matching the first image segment is obtained from the component library.

[0055] Exemplarily, the component recognition model can identify whether any of the N image segments can match a component. Thus, the first recognition result obtained after performing image recognition on the N image segments using the component recognition model can have the following three possibilities: First, each of the N image segments can match a component. In this case, the component corresponding to each image segment can be obtained from the component library and determined as the components corresponding to the N image segments; Second, a portion of the N image segments can match a component, that is, the first image segment of the N image segments can match a component. In this case, the first component matching the first image segment can be obtained from the component library and determined as the components corresponding to the N image segments. Here, another portion of the N image segments cannot match a component, and then steps 1203 to 1206 can be executed; Third, none of the N image segments can match a component, and then steps 1203 to 1206 can still be executed.

[0056] In one or more other possible embodiments, the component corresponding to the N image segments includes a second component matching a first sub-image segment of a second image segment in the N image segments. Based on this, step 120 may specifically include:

[0057] Step 1203: If the first recognition result includes a second image segment that fails to match a component among the N image segments, perform image processing on the second image segment to obtain M sub-image segments.

[0058] Step 1204: Perform image recognition on each of the M sub-image segments using a component recognition model to obtain a second recognition result.

[0059] Step 1205 : When the second recognition result includes a first sub-image segment that can match a component among the M sub-image segments, a second component matching the first sub-image segment is obtained from the component library.

[0060] For example, with respect to the exemplary second and third aspects mentioned in the above steps 1201 to 1202, if there is a second image segment among the N image segments that fails to match a component, the second image segment can be further divided through the image processing process to obtain M sub-image segments, and the M sub-image segments obtained by further division can be further identified through the component recognition model. At this time, the following three possibilities still occur: first, each of the M sub-image segments can be matched to a component. At this time, these matched components can also be combined with the first component matched to the first image segment. as components corresponding to the N image fragments; secondly, a part of the M sub-image fragments can match the components, that is, the M sub-image fragments include a first sub-image fragment that can match the component. At this time, the second components matching the first sub-image fragment can be obtained from the component library, and these second components are determined as components corresponding to the N image fragments. Here, another part of the M sub-image fragments cannot be matched to the components, then the following step 1206 can be executed; thirdly, each image fragment in the M sub-image fragments cannot be matched to the component, then the following step 1206 can also be executed.

[0061] In one or more possible embodiments, the components corresponding to the N image segments include a preset third component. Based on this, step 120 may specifically include:

[0062] Step 1206 : When the second recognition result includes a second sub-image segment in which no component is matched in at least two sub-image segments, a preset third component is used as the component corresponding to the N image segments.

[0063] For example, if no matching component is found in the further divided sub-image segments, the preset third component may be used as N components corresponding to the N image segments.

[0064] Furthermore, in one or more possible embodiments, step 1203 may specifically include:

[0065] Step 12031, performing binarization processing on the second image segment to obtain a binarized image segment;

[0066] Step 12032: Perform enhancement processing on the binary image segment using a histogram equalization algorithm to obtain a histogram equalized image segment.

[0067] Step 12033: performing dilation and erosion processing on the histogram equalization image segment to obtain a dilated and eroded image segment;

[0068] Step 12034: Process the dilated and eroded image segments using an edge detection algorithm to obtain edge-detected image segments, where the edge-detected image segments include edge sub-regions.

[0069] Step 12035: cutting the edge detection image segment along the first direction of the edge sub-region to obtain at least two rows of sub-regions;

[0070] Step 12036: cutting each row of at least two rows of sub-regions along the second direction of the edge sub-region to obtain at least two columns of sub-regions;

[0071] Step 12037: When each sub-region in at least two columns of sub-regions does not include a blank region other than an edge sub-region, determine the sub-region as a sub-image segment among the M sub-image segments.

[0072] Exemplarily, the second image segment is binarized to obtain a binarized image segment; then, the edge subregion in the binarized image segment is enhanced using a histogram equalization algorithm to obtain a histogram equalized image segment; further, to make the edge subregion in the image segment clearer, the histogram equalized image segment can be dilated and eroded to obtain a dilated and eroded image segment, in which the edge subregion in the dilated and eroded image segment is clearer; then, the dilated and eroded image segment is processed using an edge detection algorithm to obtain an edge detected image including the edge subregion. At this time, the edge detected image segment is cut along a first direction of the edge subregion, such as the horizontal direction, to obtain at least two rows of subregions. Then, each row of subregions in the at least two rows of subregions is cut along a second direction of the edge subregion, such as the vertical direction, to obtain at least two columns of subregions. Here, steps 12035 and 12036 are repeatedly performed until each subregion in the at least two columns of subregions does not include a blank area other than the edge subregion, and the subregion is determined as a subimage segment among the M subimage segments. Furthermore, before step 1204, the page data generation method may further include:

[0073] Acquire a training sample, where the training sample includes a sample component, a sample image segment corresponding to the sample component, and a sample sub-image segment;

[0074] The convolutional neural network is trained through training samples until the preset training conditions are met to obtain a component recognition model.

[0075] Exemplarily, the component recognition model can be obtained by training a convolutional neural network using a large number of sample components, sample image segments corresponding to the sample components, and training samples consisting of sample sub-image segments.

[0076] Based on this, after step 120, the page data generation method may further include:

[0077] Step 1501: Add the second sub-image segment to the training sample of the component recognition model, and train the component recognition model with the training sample until a preset training condition is met, thereby obtaining an updated component recognition model.

[0078] In this way, image segments that do not match components can be used as training samples to train the component recognition model, thereby achieving the purpose of automatically expanding the component recognition model and improving the recognition accuracy of the component recognition model.

[0079] In one example, the training sample further includes a target unknown component corresponding to the second sub-image segment. Based on this, before step 1501, the page data generation method may further include:

[0080] Step 1502: binarize the second sub-image segment to obtain a binarized sub-image segment.

[0081] Step 1503 , calculating a similarity value between the binarized sub-image segment and a preset binarized sub-image segment corresponding to each unknown component in the preset unknown component library;

[0082] Step 1504, when the similarity value between the binarized sub-image segment and the preset binarized sub-image segment of the target unknown component in the preset unknown component library is greater than or equal to a first preset threshold, and the repetition of the target unknown component is greater than or equal to a second preset threshold, the target unknown component is added to the training sample.

[0083] Exemplarily, the second sub-image segment is binarized to obtain a binarized sub-image segment, and then the similarity value between the binarized sub-image segment and the preset binarized sub-image segment corresponding to each unknown component in the preset unknown component library is calculated. If they are similar, they are classified, and it is determined whether the repetition of the target unknown component is greater than or equal to a second preset threshold. If the repetition of the target unknown component is greater than or equal to the second preset threshold, the target unknown component is added to the training sample.

[0084] Therefore, the component recognition model is trained as a training sample to achieve the purpose of automatically expanding the component recognition model. The component recognition model has the ability to self-expand and completes self-expansion through the unknown component labeling training process, thereby improving the accuracy of the component recognition model and increasing the accuracy and applicability of component recognition.

[0085] On the contrary, in another example, the page data generating method may further include:

[0086] Step 1505: When the similarity value between the binary sub-image segment and the preset binary sub-image segment of the target unknown component in the preset unknown component library is less than a first preset threshold, the binary sub-image segment corresponding to the second sub-image segment is stored in the preset unknown component library, and the binary sub-image segment is marked with a preset unknown identifier.

[0087] Exemplarily, if they are not similar, they are classified into a new type, stored in a preset unknown component library, and the binarized sub-image segment is marked with a preset unknown identifier.

[0088] In this way, the unknown component type library can be continuously expanded to increase the update capability of the unknown component library. The unknown component library can be automatically expanded to reduce the manual expansion operations of users.

[0089] Furthermore, regarding step 130, in one or more possible embodiments, the position information includes row-level image information, column-level image information, and absolute position information. Step 130 may specifically include:

[0090] Step 1301, calculating the row-level component information and row-level relative position information of the component, as well as the column-level component information and column-level relative position information of the component, based on the row-level image information, column-level image information, and absolute position information of the target image segment in the user interface design image;

[0091] Step 1302: Calculate alignment information based on row-level image information and column-level image information.

[0092] Step 1303 , correcting the row-level component information, row-level relative position information, column-level component information, and column-level relative position information of the component through alignment information to obtain layout information of the component.

[0093] Exemplarily, based on the segmentation process of steps 1501 to 1506 above, the row-level image information, column-level image information, and absolute position information of the target image segment in the user interface design image are obtained. Based on the row-level image information and column-level image information, the row-level component information and row-level relative position information of the component, as well as the column-level component information and column-level relative position information of the component, are calculated. Alignment information is then calculated and obtained in combination with the row-level image information and column-level image information. Using the alignment information, the row-level component information, row-level relative position information, column-level component information, and column-level relative position information of the component are corrected to obtain the component layout information. In one or more other possible embodiments, based on steps 1203 to 1205, the image segment is divided into sub-image segments, i.e., the target image segment may include a second sub-image segment. In this case, steps 1301 to 1303 may still be performed with the target image segment specifically being the second sub-image segment to obtain the layout information of the component corresponding to the second sub-image judgment. Therefore, the process of obtaining the layout information corresponding to the second sub-image fragment is located above the layout of obtaining the layout information corresponding to the image fragment, thereby obtaining the layout hierarchy information. The layout information obtained in the two stages together constitute the layout results of each component in the user interface design image.

[0094] Then, step 140 is involved. In one or more possible embodiments, step 140 may specifically include:

[0095] Step 1401, according to the component, obtain component development data corresponding to the component;

[0096] Step 1402: Arrange component development data corresponding to the components according to the layout information of the components to obtain page development data.

[0097] Therefore, the component development data of each component can be arranged according to the component layout information to obtain page development data. In this way, the page development data can be run to obtain a user interface corresponding to the user interface design image. In this way, the page development data can be quickly generated, and the page development data can be efficiently run.

[0098] In summary, the embodiments of the present application provide a page data generation method for user interface design image analysis based on image technology and artificial intelligence technology, which can quickly segment and analyze user interface design images to obtain component and layout information.

[0099] It should be noted that the method provided in the embodiment of the present application is suitable for building a low-code platform for mobile applications and Internet web pages, quickly generating page development data, reducing the page investment of R&D personnel, freeing R&D personnel from repetitive page construction work, reducing manual editing operations, quickly realizing business needs, shortening page development time, and improving page development efficiency.

[0100] Based on the same inventive concept, the present application also provides a page data generation device, which will be described in detail with reference to FIG2 .

[0101] FIG2 is a schematic diagram of the structure of a page data generating device provided by an embodiment of the present application.

[0102] In some embodiments of the present application, the page data generating device shown in FIG. 2 may be provided in a computer device of an embodiment of the present application.

[0103] As shown in FIG2 , the page data generating device 20 may specifically include:

[0104] An acquisition module 201 is configured to acquire a user interface design image, where the user interface design image includes N image segments, where N is a positive integer.

[0105] A determination module 202 is configured to determine components corresponding to the N image segments based on the N image segments;

[0106] A generating module 203, configured to generate layout information of the components according to position information of target image segments corresponding to the components in the user interface design image;

[0107] The generation module 203 is further configured to generate page development data based on the components and their layout information, where the page development data is used to generate a user interface corresponding to the user interface design image.

[0108] The following is a detailed description of the page data generating device 20 in the embodiment of the present application.

[0109] In one or more optional embodiments, the components corresponding to the N image segments include at least one of the following:

[0110] A first component matching a first image segment among the N image segments, a second component matching a first sub-image segment of a second image segment among the N image segments, and a preset third component.

[0111] In another or more optional embodiments, the page data generating device 20 in the embodiment of the present application may further include a first identification module; wherein,

[0112] a first recognition module configured to, when the components corresponding to the N image segments include a first component matching the first image segment, perform image recognition on the N image segments using a component recognition model to obtain a first recognition result;

[0113] The acquisition module 201 may also be configured to, when the first identification result includes a first image segment that can match a component among the N image segments, acquire a first component that matches the first image segment from the component library.

[0114] In one or more optional embodiments, the page data generating device 20 in the embodiment of the present application may further include a first processing module and a second identification module; wherein,

[0115] a first processing module configured to, when the components corresponding to the N image segments include a second component that matches a first sub-image segment of a second image segment among the N image segments, and when the first recognition result includes the second image segment that fails to match a component among the N image segments, perform image processing on the second image segment to obtain M sub-image segments, where M is a positive integer;

[0116] a second recognition module, configured to perform image recognition on each of the M sub-image segments using a component recognition model to obtain a second recognition result;

[0117] The acquisition module 201 may also be configured to, when the second recognition result includes a first sub-image segment that can match a component among the M sub-image segments, acquire a second component that matches the first sub-image segment from the component library.

[0118] In one or more optional embodiments, the determination module 202 can be specifically used to, when the components corresponding to the N image fragments include a preset third component and the second recognition result includes a second sub-image fragment that cannot be matched to a component in the M sub-image fragments, use the preset third component as the component corresponding to the N image fragments.

[0119] In one or more optional embodiments, the page data generating device 20 in the embodiment of the present application may further include a first training module; wherein,

[0120] The first training module is used to add the second sub-image segment to the training sample of the component recognition model, and train the component recognition model through the training sample until a preset training condition is met to obtain an updated component recognition model.

[0121] In one or more optional embodiments, the page data generating device 20 in the embodiment of the present application may further include a second processing module, a first calculation module and an adding module; wherein,

[0122] a second processing module, configured to perform binarization processing on the second sub-image segment to obtain a binarized sub-image segment when the training sample further includes a target unknown component corresponding to the second sub-image segment;

[0123] A first calculation module is used to calculate a similarity value between the binary sub-image segment and a preset binary sub-image segment corresponding to each unknown component in the preset unknown component library;

[0124] An adding module is used to add the target unknown component to the training sample when the similarity value between the binarized sub-image segment and the preset binarized sub-image segment of the target unknown component in the preset unknown component library is greater than or equal to a first preset threshold, and the repetition of the target unknown component is greater than or equal to a second preset threshold.

[0125] In one or more optional embodiments, the page data generating device 20 in the embodiment of the present application may further include a third processing module; wherein,

[0126] The third processing module is used to store the binary sub-image segment corresponding to the second sub-image segment in the preset unknown component library when the similarity value between the binary sub-image segment and the preset binary sub-image segment of the target unknown component in the preset unknown component library is less than a first preset threshold, and mark the binary sub-image segment with a preset unknown identifier.

[0127] In one or more optional embodiments, the page data generating device 20 in the embodiment of the present application may further include a fourth processing module, a fifth processing module, a sixth processing module, a seventh processing module, a first cutting module and a second cutting module; wherein,

[0128] a fourth processing module, configured to perform binarization processing on the second image segment to obtain a binarized image segment;

[0129] a fifth processing module, configured to perform enhancement processing on the binarized image segment by using a histogram equalization algorithm to obtain a histogram equalized image segment;

[0130] a sixth processing module, configured to perform dilation and erosion processing on the histogram equalized image segment to obtain a dilated and eroded image segment;

[0131] a seventh processing module, configured to process the dilated and eroded image segments using an edge detection algorithm to obtain edge detection image segments, wherein the edge detection image segments include edge sub-regions;

[0132] A first cutting module is used to cut the edge detection image segment along a first direction of the edge sub-region to obtain at least two rows of sub-regions;

[0133] A second cutting module is configured to cut each row of sub-regions in at least two rows of sub-regions along a second direction of the edge sub-region to obtain at least two columns of sub-regions;

[0134] The determination module may also be configured to, when each sub-region in the at least two columns of sub-regions does not include a blank region other than an edge sub-region, determine the sub-region as a sub-image segment among the M sub-image segments.

[0135] In one or more optional embodiments, the page data generating device 20 in the embodiment of the present application may further include a second training module; wherein,

[0136] The acquisition module 201 may also be used to acquire a training sample, where the training sample includes a sample component, a sample image segment corresponding to the sample component, and a sample sub-image segment;

[0137] The second training module is used to train the convolutional neural network through training samples until the preset training conditions are met to obtain a component recognition model.

[0138] In one or more optional embodiments, the page data generating device 20 in the embodiment of the present application may further include an arrangement module; wherein,

[0139] The acquisition module 201 may also be used to acquire component development data corresponding to the component according to the component;

[0140] The arrangement module is used to arrange the component development data corresponding to the components according to the layout information of the components to obtain page development data.

[0141] In one or more optional embodiments, the page data generating device 20 in the application embodiment may further include an eighth processing module; wherein,

[0142] The eighth processing module is used to perform image preprocessing on the user interface design image to obtain N image segments.

[0143] In one or more optional embodiments, the page data generating device 20 in the application embodiment may further include a ninth processing module, a tenth processing module, an eleventh processing module, a twelfth processing module, a positioning module, and a segmentation module; wherein,

[0144] a ninth processing module, configured to perform binarization processing on the user interface design image to obtain a binarized image;

[0145] a tenth processing module, configured to perform enhancement processing on the binary image by using a histogram equalization algorithm to obtain a histogram equalized image;

[0146] an eleventh processing module, configured to perform dilation and erosion processing on the histogram equalization image to obtain a dilated and eroded image;

[0147] a twelfth processing module, configured to process the dilated and eroded images using an edge detection algorithm to obtain an edge detection image, wherein the edge detection image includes an edge region;

[0148] A positioning module is used to locate closed areas in edge detection images by flood filling and edge regions;

[0149] The segmentation module is used to segment N image segments from the user interface design image according to closed areas.

[0150] In one or more optional embodiments, the page data generating device 20 in the application embodiment may further include a second calculation module, a third calculation module and a correction module; wherein,

[0151] a second calculation module, configured to calculate, when the position information includes row-level image information, column-level image information, and absolute position information, row-level component information and row-level relative position information of the component, as well as column-level component information and column-level relative position information of the component based on the row-level image information, column-level image information, and absolute position information of the target image segment in the user interface design image;

[0152] a third calculation module, configured to calculate alignment information based on the row-level image information and the column-level image information;

[0153] The correction module is used to correct the row-level component information, row-level relative position information, column-level component information and column-level relative position information of the component through alignment information to obtain the layout information of the component.

[0154] Thus, a user interface design image can be obtained, the user interface design image including N image segments; based on the N image segments, components corresponding to the N image segments are determined; according to the position information of the target image segments corresponding to the components in the user interface design image, layout information of the components is generated; based on the components and the component layout information, page development data is generated, and the page development data is used to generate a user interface corresponding to the user interface design image. In this way, by analyzing the user interface design image, information on each component in the user interface design image and the layout information of the components are obtained, and based on the components and the component layout information, page development data for generating a user interface corresponding to the user interface design image is generated. This eliminates the need for users to manually edit page code, reduces user participation, shortens page development time, and improves page development efficiency.

[0155] Based on the same inventive concept, the present application also provides a computer device, which will be described in detail with reference to FIG3 .

[0156] FIG3 is a schematic diagram of the structure of a computer device provided in one embodiment of the present application.

[0157] As shown in Figure 3, the computer device may include at least one of the following involved in the embodiments of the present application: an electronic device, a server. The computer device may include a processor 301 and a memory 302 storing computer program instructions.

[0158] Specifically, the processor 301 may include a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0159] Memory 302 may include a large capacity memory for data or instructions. By way of example and not limitation, memory 302 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 302 may include removable or non-removable (or fixed) media. Where appropriate, memory 302 may be inside or outside the computer device. In a specific embodiment, memory 302 is a non-volatile solid-state memory. In a specific embodiment, memory 302 includes solid-state storage (ROM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically rewritable ROM (EAROM), or a flash memory, or a combination of two or more of these.

[0160] The processor 301 reads and executes computer program instructions stored in the memory 302 to implement any one of the page data generation methods in the above embodiments.

[0161] In one example, the computer device may further include a communication interface 303 and a bus 310. As shown in FIG3, the processor 301, the memory 302, and the communication interface 303 are connected via the bus 310 and communicate with each other.

[0162] The communication interface 303 is mainly used to implement communication between various modules, devices, units and / or equipment in the embodiments of the present application.

[0163] Bus 310 comprises hardware, software or both, and the parts of flow control device are coupled to each other.For example, and not limitation, bus can comprise accelerated graphics port (AGP) or other graphics buses, enhanced industrial standard architecture (EISA) bus, front side bus (FSB), hypertransport (HT) interconnection, industrial standard architecture (ISA) bus, infinite bandwidth interconnection, low pin count (LPC) bus, memory bus, micro channel architecture (MCA) bus, peripheral component interconnection (PCI) bus, PCI-Express (PCI-X) bus, serial advanced technology attachment (SATA) bus, video electronics standard association local (VLB) bus or other suitable bus or two or more of these combinations.In suitable cases, bus 310 can comprise one or more buses.Although the present application embodiment describes and shows specific bus, the application considers any suitable bus or interconnection.

[0164] The data processing device can execute the page data generation method in the embodiment of the present application, thereby realizing the page data generation method and apparatus described in conjunction with FIG. 1 .

[0165] In addition, in conjunction with the page data generation method in the above embodiments, embodiments of the present application may provide a computer-readable storage medium for implementation. The computer-readable storage medium stores computer program instructions; when the computer program instructions are executed by a processor, any of the page data generation methods in the above embodiments is implemented.

[0166] It should be understood that the present application is not limited to the specific configurations and processes described above and illustrated in the figures. For the sake of brevity, a detailed description of known methods is omitted here. In the above embodiments, several specific steps are described and illustrated as examples. However, the method process of the present application is not limited to the specific steps described and illustrated. Those skilled in the art can make various changes, modifications, and additions, or change the order of the steps after understanding the spirit of the present application.

[0167] The functional blocks shown in the above block diagram can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of the present application are programs or code segments that are used to perform the required tasks. Programs or code segments can be stored in machine-readable media, or transmitted on a transmission medium or a communication link by a data signal carried in a carrier wave. "Machine-readable media" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROMs, flash memories, erasable ROMs (EROMs), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, etc. The code segments can be downloaded via computer networks such as the Internet, intranets, etc.

[0168] It should also be noted that the exemplary embodiments mentioned in this application describe some methods or systems based on a series of steps or devices. However, this application is not limited to the order of the above steps. In other words, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.

[0169] The above is only a specific implementation method of the present application. Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the systems, modules and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. It should be understood that the scope of protection of the present application is not limited to this. Any technician familiar with this technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in this application, and these modifications or replacements should be included in the scope of protection of this application.

Claims

1. A method for generating page data, comprising: Acquire a user interface design image, where the user interface design image includes N image segments, where N is a positive integer; Based on the N image segments, determining components corresponding to the N image segments; Generate layout information of the component according to position information of the target image segment corresponding to the component in the user interface design image; Page development data is generated according to the components and the layout information of the components, and the page development data is used to generate a user interface corresponding to the user interface design image.

2. The method according to claim 1, wherein: The components corresponding to the N image segments include at least one of the following: A first component matching a first image segment among the N image segments, a second component matching a first sub-image segment of a second image segment among the N image segments, and a preset third component.

3. The method according to claim 2, wherein: The components corresponding to the N image segments include a first component matched with the first image segment; and determining the components corresponding to the N image segments based on the N image segments includes: Performing image recognition on the N image segments through a component recognition model to obtain a first recognition result; In a case where the first identification result includes a first image segment that can be matched to a component among the N image segments, a first component matching the first image segment is acquired from a component library.

4. The method according to claim 3, wherein: The components corresponding to the N image segments include a second component matched with a first sub-image segment of a second image segment among the N image segments; and determining the components corresponding to the N image segments based on the N image segments includes: When the first recognition result includes a second image segment that fails to match a component among the N image segments, performing image processing on the second image segment to obtain M sub-image segments, where M is a positive integer; For each of the M sub-image segments, the component recognition model is used. Performing image recognition to obtain a second recognition result; In a case where the second recognition result includes a first sub-image segment that can be matched to a component among the M sub-image segments, a second component that matches the first sub-image segment is acquired from the component library.

5. The method according to claim 4, wherein: The components corresponding to the N image segments include a preset third component; and determining the components corresponding to the N image segments based on the N image segments includes: In a case where the second recognition result includes a second sub-image segment that fails to match a component among the M sub-image segments, a preset third component is used as a component corresponding to the N image segments.

6. The method according to claim 5, wherein: The method further comprises: The second sub-image segment is added to the training sample of the component recognition model, and the component recognition model is trained by the training sample until a preset training condition is met, thereby obtaining an updated component recognition model.

7. The method according to claim 6, wherein: The training sample also includes a target unknown component corresponding to the second sub-image segment; before training the component recognition model using the training sample, the method includes: performing binarization processing on the second sub-image segment to obtain a binarized sub-image segment; Calculating a similarity value between the binary sub-image segment and a preset binary sub-image segment corresponding to each unknown component in a preset unknown component library; When the similarity value between the binary sub-image segment and the preset binary sub-image segment of the target unknown component in the preset unknown component library is greater than or equal to a first preset threshold, and the repetition degree of the target unknown component is greater than or equal to a second preset threshold, the target unknown component is added to the training sample.

8. The method according to claim 7, wherein: The method further comprises: When the similarity value between the binary sub-image segment and the preset binary sub-image segment of the target unknown component in the preset unknown component library is less than a first preset threshold, the binary sub-image segment corresponding to the second sub-image segment is stored in the preset unknown component library, and the binary sub-image segment is marked with a preset unknown identifier.

9. The method according to claim 4, wherein: The performing image processing on the second image segment to obtain M sub-image segments includes: performing binarization processing on the second image segment to obtain a binarized image segment; Performing enhancement processing on the binary image segment by using a histogram equalization algorithm to obtain a histogram equalized image segment; Performing dilation and erosion processing on the histogram equalization image segment to obtain a dilated and eroded image segment; Processing the dilated and eroded image segments by an edge detection algorithm to obtain edge detection image segments, wherein the edge detection image segments include edge sub-regions; Cutting the edge detection image segment along a first direction of the edge sub-region to obtain at least two rows of sub-regions; Cutting each row of sub-regions in the at least two rows of sub-regions along the second direction of the edge sub-region to obtain at least two columns of sub-regions; In the case that each sub-region in the at least two columns of sub-regions does not include a blank region other than an edge sub-region, the sub-region is determined as a sub-image segment among the M sub-image segments.

10. The method according to claim 3, wherein: The method further comprises: Acquire a training sample, wherein the training sample includes a sample component, a sample image segment corresponding to the sample component, and a sample sub-image segment; The convolutional neural network is trained using the training samples until preset training conditions are met, thereby obtaining the component recognition model.

11. The method according to claim 1, wherein: The generating page development data according to the component and the layout information of the component includes: According to the component, obtaining component development data corresponding to the component; According to the layout information of the components, the component development data corresponding to the components are arranged to obtain the page development data.

12. The method according to claim 1, wherein: The method further comprises: Perform image preprocessing on the user interface design image to obtain the N image segments.

13. The method according to claim 12, wherein: The performing image preprocessing on the user interface design image to obtain the N image segments includes: Binarizing the user interface design image to obtain a binary image; The binary image is enhanced by a histogram equalization algorithm to obtain a histogram equalized image; Performing dilation and erosion processing on the histogram equalization image to obtain a dilated and eroded image; Processing the dilated and eroded images by an edge detection algorithm to obtain an edge detection image, wherein the edge detection image includes an edge region; Locating a closed area in the edge detection image by flood filling and the edge area; According to the closed area, N image segments are segmented from the user interface design image.

14. The method according to claim 1, wherein: The position information includes row-level image information, column-level image information and absolute position information; the generating the layout information of the component according to the position information of the target image segment corresponding to the component in the user interface design image includes: Calculate the row-level component information and row-level relative position information of the component, and the column-level component information and column-level relative position information of the component according to the row-level image information, column-level image information and absolute position information of the target image segment in the user interface design image; Calculating alignment information based on the row-level image information and the column-level image information; The row-level component information, the row-level relative position information, the column-level component information and the column-level relative position information of the component are corrected by using the alignment information to obtain the layout information of the component.

15. A page data generating device, comprising: An acquisition module, used for acquiring a user interface design image, wherein the user interface design image includes N image segments, where N is a positive integer; A determination module, configured to determine components corresponding to the N image segments based on the N image segments; A generating module, used for generating layout information of the component according to position information of the target image segment corresponding to the component in the user interface design image; The generating module is further used to generate page development data according to the components and the layout information of the components, wherein the page development data is used to generate a user interface corresponding to the user interface design image.

16. A computer device, the device comprising: a processor and a memory storing computer program instructions; When the processor executes the computer program instructions, the page data generating method according to any one of claims 1 to 14 is implemented.

17. A storage medium having computer program instructions stored thereon, wherein the computer program instructions, when executed by a processor, implement the page data generating method according to any one of claims 1 to 14.

Images