Front-end code template generation method, device and medium based on image detection

Abstract

The embodiments of the present disclosure disclose a method, device and medium for generating a front-end code template based on image detection. A specific implementation of the method includes: performing image detection on the target page image to generate page image detection information; determining the basic component corresponding to the maximum image similarity in the image similarity group as the target basic component; monitoring the mouse box selection operation of the target page image on the user interaction page to determine the circled range corresponding to the box selection operation; using a screenshot tool to capture the circled image corresponding to the circled range; determining the target component image; determining all coordinate information and corresponding component information covered within the circled range; generating a front-end page code template text based on the target component image, all coordinate information and corresponding component information. This implementation improves the traditional solution of automatically generating code using static templates, receives the user's incoming demand prototype diagram or UI design diagram, and integrates the code template in the pre-generated component template knowledge base.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the field of computers, and in particular to a method, device, and medium for generating a front-end code template based on image detection. Background Art

[0002] The generation of front-end code includes: static code skeleton templates and dynamic configuration data; the static code skeleton templates are: code templates of several pages and component types, stored in the cloud or locally; the dynamic configuration data are: json configuration data with other information (interface information, field information, etc.), stored in the database, and can be obtained through the back-end interface. At present, the selection of front-end code templates is usually done by directly manually selecting static templates according to business types, or receiving multi-modal input parameters, and using technologies including image position detection to dynamically match and integrate component templates.

[0003] However, the above method usually has the following technical problems: manually selecting static templates has poor flexibility and small coverage, and specific template files must be written in advance for each business type; matching component templates through image detection technology, currently mostly by calculating the similarity of component images with the UI component knowledge base to obtain the basic UI layout of the page, but cannot obtain the dynamic interaction operations and functional linkage relationships between components. The generated page template cannot be put into use directly, and developers are required to continue to develop functional logic, resulting in a decrease in the effectiveness of code generation tools.

[0004] The above information disclosed in this Background section is only for enhancement of understanding of the background of the inventive concept and therefore it may contain information that does not form the prior art that is already known in this country to a person of ordinary skill in the art. Summary of the invention

[0005] The content of this disclosure is used to introduce concepts in a brief form, which will be described in detail in the detailed implementation section below. The content of this disclosure is not intended to identify the key features or essential features of the technical solution claimed for protection, nor is it intended to limit the scope of the technical solution claimed for protection.

[0006] Some embodiments of the present disclosure propose a front-end code template generation method, device, electronic device and computer-readable medium based on image detection to solve one or more of the technical problems mentioned in the above background technology section.

[0007] In a first aspect, some embodiments of the present disclosure provide a front-end code template generation method based on image detection, the method comprising: performing image detection on a target page image to generate page image detection information, wherein the page image detection information comprises: a page detection image; determining the image similarity between the page detection image and each basic component image in a pre-established basic component image library to obtain an image similarity group; determining the basic component corresponding to the maximum image similarity in the image similarity group as the target basic component; storing the target basic component and the corresponding position coordinate information in a temporary memory in the form of a mapping object; in response to detecting a user's click operation on a selection box mode on a user interaction page, monitoring the mouse on the target on the user interaction page; A frame selection operation is performed on the page diagram to determine a circled range corresponding to the frame selection operation; a framed image corresponding to the above circled range is captured by a screenshot tool, and the above circled image is stored in the above temporary memory; the similarity between the above circled image and each component image in a pre-established component image resource library is determined to obtain a similarity group; a target component image is determined based on the above similarity group; the above circled range is compared with the above position coordinate information; in response to determining that the above circled range contains a coordinate range corresponding to the above position coordinate information, all coordinate information and corresponding component information covered within the above circled range are determined; a front-end page code template text corresponding to the above target page diagram is generated based on the above target component image, all the above coordinate information and corresponding component information.

[0008] In a second aspect, some embodiments of the present disclosure provide a front-end code template generation device based on image detection, the device comprising: a detection unit, configured to perform image detection on a target page image to generate page image detection information, wherein the page image detection information comprises: a page detection image; a first determination unit, configured to determine the image similarity between the page detection image and each basic component image in a pre-established basic component image library to obtain an image similarity group; a second determination unit, configured to determine the basic component corresponding to the maximum image similarity in the image similarity group as the target basic component; a storage unit, configured to store the target basic component and the corresponding position coordinate information in a temporary memory in the form of a mapping object; a monitoring unit, configured to monitor the box selection of the target page image on the user interaction page by the mouse in response to detecting a click operation of the user on the selection box mode on the user interaction page. operation to determine the circled range corresponding to the box selection operation; a capture unit is configured to capture the circled image corresponding to the above-mentioned circled range through a screenshot tool, and store the above-mentioned circled image in the above-mentioned temporary memory; a third determination unit is configured to determine the similarity between the above-mentioned circled image and each component image in a pre-established component image resource library to obtain a similarity group; a fourth determination unit is configured to determine the target component image according to the above-mentioned similarity group; a comparison unit is configured to compare the above-mentioned circled range with the above-mentioned position coordinate information; a fifth determination unit is configured to determine all the coordinate information and the corresponding component information covered in the above-mentioned circled range in response to determining that the above-mentioned circled range contains the coordinate range corresponding to the above-mentioned position coordinate information; a generation unit is configured to generate a front-end page code template text corresponding to the above-mentioned target page image according to the above-mentioned target component image, the above-mentioned all the coordinate information and the corresponding component information.

[0009] In a third aspect, some embodiments of the present disclosure provide an electronic device comprising: one or more processors; a storage device on which one or more programs are stored, and when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation manner of the above-mentioned first aspect.

[0010] In a fourth aspect, some embodiments of the present disclosure provide a computer-readable medium having a computer program stored thereon, wherein when the program is executed by a processor, the method described in any implementation manner of the above-mentioned first aspect is implemented.

[0011] The above-mentioned embodiments of the present disclosure have the following beneficial effects: through the front-end code template generation method based on image detection of some embodiments of the present disclosure, the traditional solution of automatically generating code using static templates is improved, the user's incoming demand prototype or UI design drawing is received, and the code template is integrated in the pre-generated component template knowledge base. In addition, a resource library that stores the mapping relationship between component code and UI image is designed, and the resource library simultaneously stores a variety of complex components encapsulated in the front end, and establishes a mapping with the UI appearance images presented by these components. The user continues to circle and narrow the range on the input image through the image selection tool, compares the component images stored in the resource library, and selects one of the first few complex component images with a higher similarity ranking, so as to more accurately replace the code template in the area, so as to improve the accuracy and effectiveness of the result of automatically generating code through the template. First, the target page image is image detected to generate page image detection information. Among them, the above-mentioned page image detection information includes: page detection image. As a result, it is convenient to detect the image that needs to be replaced in the above-mentioned target page image. Next, determine the image similarity between the page detection image and each basic component image in the pre-established basic component image library to obtain an image similarity group; determine the basic component corresponding to the maximum image similarity in the image similarity group as the target basic component; store the target basic component and the corresponding position coordinate information in a temporary memory in the form of a mapping object; in response to detecting the user's click operation on the selection box mode on the user interaction page, monitor the mouse's box selection operation on the target page image on the user interaction page to determine the circled range corresponding to the box selection operation; capture the circled image corresponding to the circled range through a screenshot tool, and store the circled image in the temporary memory; determine the similarity between the circled image and each component image in the pre-established component image resource library to obtain a similarity group; determine the target component image based on the similarity group. Compare the circled range with the position coordinate information. Thus, the component images stored in the resource library can be compared, and one of the first few complex component images with higher similarity rankings can be selected, so as to more accurately replace the code template in the area to improve the accuracy and effectiveness of the code results automatically generated by the template. Finally, in response to determining that the above-circled range includes the coordinate range corresponding to the above-circled coordinate information, all the coordinate information and corresponding component information covered in the above-circled range are determined; based on the above-mentioned target component image, all the above-mentioned coordinate information and corresponding component information, a front-end page code template text corresponding to the above-mentioned target page image is generated. In this way, the traditional solution of automatically generating code using static templates is improved, and the user's demand prototype diagram or UI design diagram is received, and the code template is integrated in the pre-generated component template knowledge base. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that components and elements are not necessarily drawn to scale.

[0013] Figure 1 is a flowchart of some embodiments of the method for generating a front-end code template based on image detection according to the present disclosure;

[0014] Figure 2 It is a structural schematic diagram of some embodiments of the front-end code template generation device based on image detection according to the present disclosure;

[0015] Figure 3 It is a schematic diagram of the structure of an electronic device suitable for implementing some embodiments of the present disclosure. DETAILED DESCRIPTION

[0016] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0017] It should also be noted that, for ease of description, only the parts related to the invention are shown in the drawings. In the absence of conflict, the embodiments and features in the embodiments of the present disclosure can be combined with each other.

[0018] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0019] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0020] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0021] The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0022] Figure 1The flowchart of some embodiments of the method for generating a front-end code template based on image detection according to the present disclosure is shown. The process 100 of some embodiments of the method for generating a front-end code template based on image detection according to the present disclosure is shown. The method for generating a front-end code template based on image detection includes the following steps:

[0023] Step 101, perform image detection on the target page image to generate page image detection information. In some embodiments, the execution subject (e.g., a computing device) of the front-end code template generation method based on image detection can perform image detection on the target page image to generate page image detection information. Wherein, the above-mentioned page image detection information includes: page detection image. For example, the target page image can be subjected to image detection by a target detection algorithm (such as a Yolo algorithm) to generate page image detection information. The target page image may refer to a page prototype image or a page design image. The page image detection information may also include border position parameter information of the image in the target page image. The border position parameter information may refer to the coordinate information of each edge of the image. The page detection image may be an image within the border of the smallest range in the identified target page image.

[0024] In practice, the above-mentioned execution entity can input the above-mentioned target page image into a pre-trained page image detection model to obtain page image detection information. The page image detection model can be a pre-trained neural network model that takes the target page image as input and takes page image detection information as output. For example, the page image detection model can be a DETR (Detection Transformer) target detection model, or it can be an R-CNN series model, a YOLO series model, an SSD model, a RetinaNet model, an EfficientDet model, a Cascade R-CNN model, and a CornerNet model.

[0025] Optionally, the page image detection model can be trained through the following steps:

[0026] The first step is to obtain a set of page image samples. The page image samples include: page images and components. That is, there is a corresponding relationship between the page image and the component. For example, the page image contains an image composed of the component.

[0027] In the second step, based on the page graph sample set, the following training steps are performed:

[0028] 1. Input each page image sample of at least one page image sample in the page image sample set into the initial page image detection model to obtain the initial page image detection result corresponding to each page image sample of the at least one page image sample. The initial page image detection result includes: an initial detection page image and an initial detection component. The initial page image detection model can be an untrained target detection model.

[0029] 2. Determine a page graph loss value between an initial detection page graph included in each page graph sample in at least one page graph sample and a corresponding page graph label. The page graph loss value between an initial detection page graph included in each page graph sample in at least one page graph sample and a corresponding page graph label may be determined by a preset loss function. For example, the preset loss function may be a hinge loss function or a cross entropy loss function.

[0030] 3. Determine a component loss value between an initial detection component included in each page graph sample in at least one page graph sample and a corresponding component label. The component loss value between an initial detection component included in each page graph sample in at least one page graph sample and a corresponding component label may be determined by a preset loss function. For example, the preset loss function may be a hinge loss function or a cross entropy loss function.

[0031] 4. Generate a model loss value based on the loss values ​​of each page graph and the component loss values. For example, the weighted sum of the loss values ​​of each page graph and the component loss values ​​can be used to obtain the model loss value.

[0032] 5. In response to determining that the above model loss value is less than or equal to the preset loss value, the initial page image detection model is determined as the trained page image detection model.

[0033] Therefore, the images and components in the page image can be accurately detected through the pre-trained page image detection model to facilitate the replacement of components.

[0034] Step 102: determine the image similarity between the page detection image and each basic component image in a pre-established basic component image library to obtain an image similarity group.

[0035] In some embodiments, the execution entity may determine the image similarity between the page detection image and each basic component image in a pre-established basic component image library to obtain an image similarity group. For example, the image similarity matching algorithm may be used to determine the image similarity between the page detection image and each basic component image in a pre-established basic component image library to obtain an image similarity group. For example, the image similarity matching algorithm may be a grayscale histogram method, a color histogram, a projection comparison, a block comparison, an average hash algorithm (aHash), a perceptual hash algorithm (pHash), a difference hash algorithm (dHash), a structural similarity algorithm (SSIM), etc.

[0036] Step 103: determine the basic component corresponding to the maximum image similarity in the above image similarity group as the target basic component.

[0037] In some embodiments, the execution entity may determine the basic component corresponding to the maximum image similarity in the image similarity group as the target basic component.

[0038] Step 104: store the target basic component and the corresponding position coordinate information in a temporary memory in the form of a mapping object.

[0039] In some embodiments, the execution subject may store the target basic component and the corresponding position coordinate information in a temporary memory in the form of a mapping object. The position coordinate information may refer to the position coordinates of the target basic component in the page. The component may refer to a UI component.

[0040] Step 105, in response to detecting a click operation of the user on the selection box mode on the user interaction page, monitoring a selection box operation of the mouse on the target page image on the user interaction page to determine a circle range corresponding to the selection box operation.

[0041] In some embodiments, the execution subject may, in response to detecting a click operation of a user on a selection box mode on a user interaction page, monitor a selection box operation of a mouse on a target page image on the user interaction page to determine a circled range corresponding to the selection box operation. The user interaction page may be a page that displays a target page image and is used for the user to operate the target page image. That is, the user may select a target page image on the user interaction page by using a mouse.

[0042] Step 106, using a screenshot tool, capture a circled image corresponding to the circled range, and store the circled image in the temporary memory.

[0043] In some embodiments, the execution entity may capture the circled image corresponding to the circled range through a screenshot tool, and store the circled image in the temporary memory.

[0044] Step 107 , determining the similarity between the circled image and each component image in a pre-established component image resource library, and obtaining a similarity group.

[0045] In some embodiments, the execution subject may determine the similarity between the circled image and each component image in a pre-established component image resource library to obtain a similarity group. The similarity between the circled image and each component image in a pre-established component image resource library may be determined by an image similarity matching algorithm to obtain a similarity group. For example, the image similarity matching algorithm may be a grayscale histogram method, a color histogram, a projection comparison, a block comparison, an average hash algorithm (aHash), a perceptual hash algorithm (pHash), a difference hash algorithm (dHash), a structural similarity algorithm (SSIM), etc.

[0046] Optionally, the component image resource library can be established by following the steps below:

[0047] The first step is to export the relative path of each component relative to the project root path.

[0048] The second step is to set the parameter type received by each component. The parameter type can be a js object type.

[0049] The third step is to set the component ID of each component.

[0050] The fourth step is to construct an image link for each component's corresponding image. The image link can be a URL link.

[0051] The fifth step is to capture the page image corresponding to the component on each historical project page.

[0052] The sixth step is to map each page image with the component data of the corresponding component to obtain component image mapping relationship information, and store each page image, the corresponding component and the component image mapping relationship information in the component image resource library. The component data may include: relative path, parameter type, component identifier, image link.

[0053] Therefore, a resource library that stores the mapping relationship between component code and UI image is designed. The resource library also stores a variety of complex components encapsulated by the front end, and establishes a mapping with the UI appearance images presented by these components. The user continues to circle and narrow the range on the input image through the image selection tool, compares the component images stored in the resource library, and selects one of the first few complex component images with high similarity ranking, so as to more accurately replace the code template in the area, so as to improve the accuracy and effectiveness of the code results automatically generated by the template.

[0054] Step 108: Determine the target component image based on the similarity group.

[0055] In some embodiments, the execution entity may determine the target component image based on the similarity group.

[0056] In practice, the above execution entity can determine the target component image through the following steps:

[0057] In the first step, each similarity in the above similarity group is processed in descending order to obtain a similarity sequence.

[0058] In the second step, the previous target number of similarities in the above similarity sequence is determined as a candidate similarity group.

[0059] In the third step, the component image corresponding to each candidate similarity in the candidate similarity group is determined as a candidate component image to obtain a candidate component image group.

[0060] The fourth step is to display the above-mentioned candidate component image group on the user interaction page for the user to select.

[0061] In a fifth step, in response to detecting a selection operation on any candidate component image in the above candidate component image group, the selected candidate component image is determined as a target component image.

[0062] Step 109, comparing the above-defined range with the above-mentioned position coordinate information.

[0063] In some embodiments, the execution entity may compare the defined range with the position coordinate information, that is, it may determine whether the defined range covers the position coordinates included in the position coordinate information.

[0064] Step 110, in response to determining that the circled range includes the coordinate range corresponding to the position coordinate information, determine all coordinate information and corresponding component information covered by the circled range.

[0065] In some embodiments, the execution subject may determine all coordinate information and corresponding component information covered within the circled range in response to determining that the circled range includes the coordinate range corresponding to the position coordinate information. The all coordinate information may refer to each position coordinate in the position coordinate information covered within the circled range. The corresponding component information may refer to the component information corresponding to each position coordinate.

[0066] Step 111, generating a front-end page code template text corresponding to the target page image according to the target component image, all the coordinate information and the corresponding component information.

[0067] In some embodiments, the execution entity may generate a front-end page code template text corresponding to the target page image according to the target component image, all the coordinate information and the corresponding component information.

[0068] In practice, the execution subject may generate the front-end page code template text corresponding to the target page diagram through the following steps:

[0069] The first step is to replace the component information corresponding to the target component image with the above component information.

[0070] In the second step, according to the preset code rules of the front-end component, the replaced target component image and all the above coordinate information are constructed into the front-end page code template text. The code rule can refer to the splicing rule or the construction rule of the code template. For example, the replaced target component image can be filled into the front-end page code template according to the coordinates included in all the above coordinate information to obtain the front-end page code template text.

[0071] It should be noted that the component information used above includes not only the component type, component name, component path, etc., but also all the data required to generate the code template, such as the component code, which may be stored in multiple locations such as the local front-end project, the cloud, and the database. Obtain it according to the actual project situation.

[0072] Optionally, in response to receiving a reuse request corresponding to a target reuse component in the above-mentioned component image resource library, a component identifier corresponding to the target reuse component is obtained. The target reuse component may be a component to be reused. The reuse of a component may be the generation of a component instance. A component may correspond to at least one component instance. A new component instance may be combined with other graphic elements to create a new component. For example, the target reuse component may be a navigation bar to be reused.

[0073] Optionally, a component instance corresponding to the above-mentioned target reusable component is generated. The above-mentioned component instance includes: a component instance identifier and a component identifier. The component instance identifier can represent the identity information of the component instance. For example, the component instance identifier can be "Po-Ypm". For example, a component instance corresponding to the above-mentioned target reusable component can be generated through a component instance generator. The component instance generator can execute the acquisition of the component identifier corresponding to the reusable component and the generation of the component instance identifier.

[0074] Optionally, a component processing operation is performed on the above-mentioned component instance, and the above-mentioned target reused component is recorded in the reused component library. Among them, the component processing operation can be an operation of performing instance processing on the component instance. For example, the component processing operation can be: component instance attribute change operation, component instance attribute deletion operation, component generation operation. The reused component library can be a library that performs various processing on the reused component. Various processing may include: reused component storage processing, reused component corresponding component information sending, reused component corresponding attribute information modification. Instance attribute change operation can be an operation of changing attribute related information in the instance. Attribute related information may include: attribute name and attribute content. Component instance attribute deletion operation can be an operation of deleting attribute related information in the instance. Component generation operation can be an operation of generating a corresponding component based on an instance.

[0075] Optionally, in response to receiving the first attribute content acquisition information for the component instance, it is determined whether the attribute content corresponding to the first target attribute in the component instance has been modified. The first target attribute is the attribute object corresponding to the first attribute content acquisition information. The first attribute content acquisition information may represent information on how to adjust the attribute content corresponding to the first target attribute in the component instance.

[0076] Optionally, in response to determining that the attribute content corresponding to the first target attribute has not been modified, the attribute content corresponding to the first target attribute sent by the multiplexing component library is obtained as the first target attribute content. The first target attribute content is the attribute content corresponding to the first target attribute in the target multiplexing component.

[0077] Optionally, in response to receiving the first attribute content adjustment information for the target multiplexing component, the attribute content corresponding to the second target attribute in the target multiplexing component is adjusted to obtain the first attribute adjustment content. The second target attribute is the attribute object corresponding to the first attribute content adjustment information. The first attribute content adjustment information may be adjustment information indicating how to adjust the attribute content corresponding to the second target attribute in the target multiplexing component.

[0078] Optionally, it is determined whether there is any modification to the attribute content of the second target attribute in the component instance.

[0079] Optionally, in response to determining that the attribute content corresponding to the above-mentioned second target attribute has not been modified and receiving the second attribute content acquisition information for the above-mentioned component instance, the above-mentioned first attribute adjustment content sent by the above-mentioned reusable component library is obtained as the attribute content corresponding to the above-mentioned second target attribute in the above-mentioned component instance.

[0080] Optionally, in response to determining that the attribute content corresponding to the second target attribute is modified, information indicating that the attribute content corresponding to the second target attribute in the component instance is not adjusted is generated.

[0081] Optionally, in response to receiving the second attribute content adjustment information for the component instance, the attribute content corresponding to the third target attribute in the component instance is adjusted according to the second attribute content adjustment information to generate the second attribute adjustment content. The third target attribute is the attribute object corresponding to the second attribute content adjustment information. The second attribute content adjustment information may represent information on how to adjust the attribute content in the third target attribute in the component instance. For example, the execution subject may adjust the attribute content corresponding to the third target attribute in the component instance according to the attribute content adjustment logic corresponding to the second attribute content adjustment information to generate the second attribute adjustment content.

[0082] Optionally, in response to determining that the attribute content corresponding to the first target attribute is modified, the modified attribute content for the first target attribute in the component instance is obtained as the first target attribute content.

[0083] Therefore, by recording the target reused components, each reused component can be effectively managed and controlled, which facilitates the efficient management of component information related to the reused components, reduces the demand for storage space, and effectively realizes the reuse of components.

[0084] Further references Figure 2 As an implementation of the methods shown in the above figures, the present disclosure provides some embodiments of a front-end code template generation device based on image detection. These embodiments of the front-end code template generation device based on image detection are similar to Figure 1 Corresponding to the method embodiments shown, the front-end code template generation device based on image detection can be specifically applied to various electronic devices.

[0085] like Figure 2As shown, the front-end code template generation device 200 based on image detection in some embodiments includes: a detection unit 201, a first determination unit 202, a second determination unit 203, a storage unit 204, a monitoring unit 205, an interception unit 206, a third determination unit 207, a fourth determination unit 208, a comparison unit 209, a fifth determination unit 210 and a generation unit 211. Among them, the detection unit 201 is configured to perform image detection on the target page image to generate page image detection information, wherein the above-mentioned page image detection information includes: a page detection image; a first determination unit 202 is configured to determine the image similarity between the above-mentioned page detection image and each basic component image in a pre-established basic component image library to obtain an image similarity group; a second determination unit 203 is configured to determine the basic component corresponding to the maximum image similarity in the above-mentioned image similarity group as the target basic component; a storage unit 204 is configured to store the above-mentioned target basic component and the corresponding position coordinate information in a temporary memory in the form of a mapping object; a monitoring unit 205 is configured to respond to detecting a user's click operation on a selection box mode on the user interaction page, monitor the mouse's box selection operation on the target page image on the above-mentioned user interaction page, so as to determine the circle range corresponding to the box selection operation; intercept a single Element 206 is configured to capture the circled image corresponding to the above-mentioned circled range through a screenshot tool, and store the above-mentioned circled image in the above-mentioned temporary memory; the third determination unit 207 is configured to determine the similarity between the above-mentioned circled image and each component image in a pre-established component image resource library to obtain a similarity group; the fourth determination unit 208 is configured to determine the target component image based on the above-mentioned similarity group; the comparison unit 209 is configured to compare the above-mentioned circled range with the above-mentioned position coordinate information; the fifth determination unit 210 is configured to determine all the coordinate information and the corresponding component information covered in the above-mentioned circled range in response to determining that the above-mentioned circled range contains the coordinate range corresponding to the above-mentioned position coordinate information; the generation unit 211 is configured to generate a front-end page code template text corresponding to the above-mentioned target page image based on the above-mentioned target component image, the above-mentioned all the coordinate information and the corresponding component information.

[0086] It can be understood that the units recorded in the front-end code template generation device 200 based on image detection are similar to those in the reference Figure 1 Therefore, the operations, features and beneficial effects described above for the method are also applicable to the front-end code template generation device 200 based on image detection and the units contained therein, and will not be repeated here.

[0087] Reference below Figure 3, which shows a schematic diagram of the structure of an electronic device 300 (e.g., a computing device) suitable for implementing some embodiments of the present disclosure. The electronic devices in some embodiments of the present disclosure may include, but are not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 3 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0088] like Figure 3 As shown, the electronic device 300 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 301, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 302 or a program loaded from a storage device 308 into a random access memory (RAM) 303. In the RAM 303, various programs and data required for the operation of the electronic device 300 are also stored. The processing device 301, the ROM 302, and the RAM 303 are connected to each other via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0089] Typically, the following devices may be connected to the I / O interface 305: input devices 306 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 307 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 308 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 309. The communication devices 309 may allow the electronic device 300 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 3 The electronic device 300 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead. Figure 3 Each block shown in the figure may represent one device, or may represent multiple devices as required.

[0090] In particular, according to some embodiments of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, some embodiments of the present disclosure include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In some such embodiments, the computer program can be downloaded and installed from the network through the communication device 309, or installed from the storage device 308, or installed from the ROM 302. When the computer program is executed by the processing device 301, the above-mentioned functions defined in the method of some embodiments of the present disclosure are executed.

[0091] It should be noted that the computer-readable medium recorded in some embodiments of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In some embodiments of the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In some embodiments of the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0092] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0093] The above-mentioned computer-readable medium may be included in the above-mentioned electronic device; or it may exist independently without being assembled into the electronic device. The above-mentioned computer-readable medium carries one or more programs. When the above-mentioned one or more programs are executed by the electronic device, the electronic device: performs image detection on the target page image to generate page image detection information, wherein the above-mentioned page image detection information includes: page detection image; determines the image similarity between the above-mentioned page detection image and each basic component image in the pre-established basic component image library to obtain an image similarity group; determines the basic component corresponding to the maximum image similarity in the above-mentioned image similarity group as the target basic component; stores the above-mentioned target basic component and the corresponding position coordinate information in the form of a mapping object in a temporary memory; in response to detecting a user's click operation on the selection box mode on the user interaction page, monitors the mouse on the above-mentioned user interaction page; A frame selection operation is performed on the target page image on the surface to determine the circled range corresponding to the frame selection operation; a frame selection tool is used to capture the circled image corresponding to the above circled range, and the above circled image is stored in the above temporary memory; the similarity between the above circled image and each component image in the pre-established component image resource library is determined to obtain a similarity group; based on the above similarity group, a target component image is determined; the above circled range is compared with the above position coordinate information; in response to determining that the above circled range contains the coordinate range corresponding to the above position coordinate information, all the coordinate information and the corresponding component information covered in the above circled range are determined; based on the above target component image, all the above coordinate information and the corresponding component information, a front-end page code template text corresponding to the above target page image is generated.

[0094] Computer program code for performing the operations of some embodiments of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0095] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0096] The units described in some embodiments of the present disclosure may be implemented by software or by hardware. The described units may also be set in a processor, for example, may be described as: a processor including: a detection unit, a first determination unit, a second determination unit, a storage unit, a monitoring unit, an interception unit, a third determination unit, a fourth determination unit, a comparison unit, a fifth determination unit and a generation unit. Among them, the names of these units do not constitute a limitation on the unit itself under certain circumstances. For example, the generation unit may also be described as "a unit that generates a front-end page code template text corresponding to the above-mentioned target page image according to the above-mentioned target component image, the above-mentioned all coordinate information and the corresponding component information."

[0097] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0098] The above descriptions are only some preferred embodiments of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by a specific combination of the above-mentioned technical features, but should also cover other technical solutions formed by any combination of the above-mentioned technical features or their equivalent features without departing from the above-mentioned inventive concept. For example, the above-mentioned features are replaced with the technical features with similar functions disclosed in the embodiments of the present disclosure (but not limited to) and the technical solutions formed.

Claims

1. A front-end code template generation method based on image detection, comprising: Performing image detection on the target page image to generate page image detection information, wherein the page image detection information includes: a page detection image, the target page image refers to a page prototype image or a page design image, the page image detection information also includes border position parameter information of the image in the target page image, the border position parameter information refers to each coordinate information of the edge of the image, and the page detection image is the image within the smallest range of the border in the identified target page image; Determine the image similarity between the page detection image and each basic component image in a pre-established basic component image library to obtain an image similarity group; Determine the basic component corresponding to the maximum image similarity in the image similarity group as the target basic component; The target basic component and the corresponding position coordinate information are stored in a temporary memory in the form of a mapping object; In response to detecting a click operation of a user on a selection box mode on a user interaction page, monitoring a selection box operation of a mouse on a target page image on the user interaction page to determine a circled range corresponding to the selection box operation; Using a screenshot tool, capturing a circled image corresponding to the circled range, and storing the circled image in the temporary memory; Determine the similarity between the circled image and each component image in a pre-established component image resource library to obtain a similarity group; Determining a target component image according to the similarity group; Comparing the defined range with the location coordinate information; In response to determining that the circled range includes the coordinate range corresponding to the position coordinate information, determining all coordinate information and corresponding component information covered within the circled range; A front-end page code template text corresponding to the target page image is generated according to the target component image, all the coordinate information and the corresponding component information.

2. The method according to claim 1, wherein: The generating of the front-end page code template text corresponding to the target page image according to the target component image, the entire coordinate information and the corresponding component information comprises: Replacing the component information corresponding to the target component image with the component information covered within the circled range; According to the preset code rules of the front-end component, the replaced target component image and the entire coordinate information are constructed into a front-end page code template text.

3. The method according to claim 1, wherein: Determining the target component image according to the similarity group includes: Performing descending processing on each similarity in the similarity group to obtain a similarity sequence; Determine the first target number of similarities in the similarity sequence as a candidate similarity group; Determine the component image corresponding to each candidate similarity in the candidate similarity group as a candidate component image, and obtain a candidate component image group; Displaying the candidate component image group on a user interaction page for user selection; In response to detecting a selection operation on any candidate component image in the candidate component image group, the selected candidate component image is determined as a target component image.

4. The method according to claim 1, wherein: Before determining the similarity between the circled image and each component image in the pre-established component image resource library to obtain the similarity group, the method further includes: Export the relative path of each component relative to the project root path; Set the parameter type received by each component; Set the component ID of each component; Build image links for each component's corresponding image; Capture the page image corresponding to the component on each historical project page; Map each page image with the component data of the corresponding component to obtain component image mapping relationship information, and store each page image, the corresponding component and the component image mapping relationship information in a component image resource library.

5. The method according to claim 1, wherein: The method further comprises: In response to receiving a reuse request corresponding to a target reuse component in the component image resource library, obtaining a component identifier corresponding to the target reuse component; Generate a component instance corresponding to the target reused component, wherein the component instance includes: a component instance identifier and a component identifier; A component processing operation is performed on the component instance, and the target reusable component is recorded in a reusable component library.

6. The method according to claim 1, wherein: The performing image detection on the target page image to generate page image detection information includes: The target page image is input into a pre-trained page image detection model to obtain page image detection information.

7. A front-end code template generation device based on image detection, comprising: The detection unit is configured to perform image detection on the target page image to generate page image detection information, wherein the page image detection information includes: a page detection image, the target page image refers to a page prototype image or a page design image, the page image detection information also includes border position parameter information of the image in the target page image, the border position parameter information refers to each coordinate information of the edge of the image, and the page detection image is the image within the border of the smallest range in the identified target page image; A first determining unit is configured to determine the image similarity between the page detection image and each basic component image in a pre-established basic component image library to obtain an image similarity group; A second determining unit is configured to determine a basic component corresponding to a maximum image similarity in the image similarity group as a target basic component; A storage unit configured to store the target basic component and the corresponding position coordinate information in a temporary memory in the form of a mapping object; A monitoring unit is configured to monitor a mouse box selection operation on a target page image on the user interaction page in response to detecting a user's click operation on a box selection mode on the user interaction page, so as to determine a circle range corresponding to the box selection operation; A capture unit is configured to capture a circled image corresponding to the circled range through a screenshot tool, and store the circled image in the temporary memory; a third determining unit, configured to determine a similarity between the circled image and each component image in a pre-established component image resource library to obtain a similarity group; a fourth determining unit, configured to determine a target component image according to the similarity group; A comparison unit is configured to compare the demarcated range with the position coordinate information; A fifth determining unit is configured to determine all coordinate information and corresponding component information covered within the circled range in response to determining that the circled range includes the coordinate range corresponding to the position coordinate information; The generating unit is configured to generate a front-end page code template text corresponding to the target page image according to the target component image, the entire coordinate information and the corresponding component information.

8. An electronic device comprising: one or more processors; a storage device having one or more programs stored thereon; When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 6.

9. A computer readable medium having a computer program stored thereon, wherein: When the program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images