Method, system and equipment for determining image code and storage medium

Abstract

The invention provides a method, a system and equipment for determining an image code and a storage medium. The method comprises the following steps: receiving a to-be-converted page image transmitted by a client; performing grid division on the page image, and extracting features of local page images in all grid sub-regions to obtain a plurality of target sub-features; and based on a preset knowledge base, performing matching on the plurality of target sub-features, and determining a target code matched with the page image. According to the method, detail information in the page image can be more accurately captured, compared with global image matching, the local-to-local matching mode can reduce the mismatching problem caused by overall similarity of the image but large local detail difference, and when the position or style of a certain element in the page image is changed, the matching accuracy is improved. In the code generation process, only the sub-features of the grid where the element is located need to be rematched, the whole image does not need to be reprocessed, and therefore the code generation efficiency and flexibility are improved.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a method, system, device and storage medium for determining an image code. Background Art

[0002] In the current context of rapid development of enterprise informatization, with the continuous expansion and complexity of the company's business scale, the demand for various management systems and subsystems is increasing, especially those application systems focusing on data visualization, report display and large-screen monitoring, which have become key tools for improving enterprise operational efficiency and enhancing decision-making support capabilities.

[0003] In the traditional front-end development process, faced with a variety of design drafts, developers usually need to manually convert the designs into hypertext markup language, cascading style sheets and script codes. This process is not only time-consuming and labor-intensive, but also easily leads to uneven code quality due to human factors, which in turn affects the user experience and performance of the final product. Especially when dealing with complex data visualization components, dynamic interactive effects and large-screen layouts, the problem of repeatedly writing basic styles and logic codes is particularly prominent, greatly reducing development efficiency.

[0004] In addition, the overall design draft image can be matched for image similarity to determine the candidate image that matches it, and then mapped to the various control codes in the candidate image to form the corresponding target code. However, as the scale of the project expands, the iteration speed of the design draft is also accelerating. Traditional code generation methods often cannot keep up with the pace of design changes, resulting in extended development cycles and even a disconnect between design and implementation. More importantly, existing automated tools often lack sufficient flexibility and accuracy when parsing design drafts and generating code, making it difficult to meet developers' needs for customization of specific components, and are unable to effectively support subsequent code correction and preview work. Summary of the invention

[0005] One of the purposes of the present invention is to provide a method for determining an image code to solve the technical problem in the prior art that when performing code matching based on complete image similarity, local differences in the image are easily ignored, affecting the matching accuracy and matching efficiency.

[0006] One of the objects of the present invention is to provide a system for determining an image code.

[0007] One of the objects of the present invention is to provide a refrigeration device.

[0008] One of the objects of the present invention is to provide a computer storage medium.

[0009] In order to achieve one of the above-mentioned purposes of the invention, the present invention provides a method for determining an image code, comprising: receiving a page image to be converted transmitted from a client; gridding the page image, and extracting features of the local page image in all grid sub-areas to obtain a number of target sub-features; based on a preset knowledge base, matching the several target sub-features to determine a target code that matches the page image.

[0010] As a further improvement of an embodiment of the present invention, the matching of several target sub-features to determine the target code that matches the page image includes: based on a cascade matching method, determining candidate sub-features that match the target sub-features in a preset knowledge base, and determining the corresponding target code based on the candidate sub-features.

[0011] As a further improvement of an embodiment of the present invention, the method of determining candidate sub-features matching the target sub-features in a preset knowledge base based on a cascade matching method includes: determining a first candidate feature set based on the similarity between the first target sub-feature and the candidate sub-features in the preset knowledge base; determining a second candidate feature set based on the similarity between the second target sub-feature and the candidate sub-features in the first candidate feature set; and iteratively determining candidate sub-features matching other target sub-features based on the second candidate feature set until the similarities of all target sub-features meet the set conditions, thereby determining the final candidate sub-features.

[0012] As a further improvement of an embodiment of the present invention, the cascade matching method is based on determining the candidate sub-features in the preset knowledge base that match the target sub-feature, including: obtaining all data blocks in the preset knowledge base, each data block pointing to the grid division information of the corresponding candidate page image, and the grid division information including a number of candidate sub-features after the grid division of the candidate page image; constructing a corresponding index for each data block, and adding the corresponding number of candidate sub-features to the index to form a number of feature indexes; based on the feature index, performing a similarity search on the target sub-feature to determine the candidate sub-features whose similarity meets the preset conditions.

[0013] As a further improvement of an embodiment of the present invention, the matching of multiple target sub-features to determine the target code that matches the page image includes: determining the corresponding multiple candidate feature sets based on the similarity between each target sub-feature and the candidate sub-features in a preset knowledge base; and determining the corresponding target code based on the intersection of all candidate feature sets.

[0014] As a further improvement of an embodiment of the present invention, determining the corresponding target code according to the intersection of all candidate feature sets includes: merging the candidate subcodes corresponding to all candidate subfeatures in the intersection result to form the target code.

[0015] As a further improvement of an embodiment of the present invention, after determining the target code matching the page image, it includes: creating a web application, the web application including a first route and a second route, the first route is used to render a page containing the target code editing area, and the second route is used to save the editing operation of the target code; based on the first route and the second route, the target code is transmitted to the client for display and editing.

[0016] As a further improvement of an embodiment of the present invention, before the page image is gridded, the method also includes: determining whether the page image is a grayscale image; if not, converting the page image into a corresponding grayscale image; extracting features of local page images in all grid areas to obtain a number of target sub-features, including: based on a pre-trained convolutional neural network model, extracting features of each local page image to obtain a number of target sub-features.

[0017] As a further improvement of an embodiment of the present invention, before the pre-trained convolutional neural network model, the method also includes: updating the number of channels of the initial convolutional neural network model to a single channel, and adjusting the mean and variance of the single channel; training the initial convolutional neural network model based on the training data to obtain a trained convolutional neural network model.

[0018] In order to achieve one of the above-mentioned purposes of the invention, the present invention also provides a system for determining an image code, comprising: a client, used to determine a page image to be converted; a server, used to receive the page image transmitted from the client, and to grid the page image, extract the features of the local page image in all grid sub-areas, and obtain a number of target sub-features; and used to perform matching on a number of target sub-features based on a preset knowledge base to determine a target code that matches the page image.

[0019] As a further improvement of an embodiment of the present invention, the target code is transmitted to the display area of ​​the client for editing and / or display, and the display area includes a first area and a second area; the first area is used to display the target code, and when it is detected that the target code in the first area is modified, the second area displays the corresponding modified target control.

[0020] To achieve one of the above-mentioned purposes of the invention, the present invention also provides a refrigeration device, comprising: at least one processor; a memory, wherein the memory stores a computer program that can be run on the processor, and when the processor executes the program, it executes any step of the method for determining the image code.

[0021] To achieve one of the above-mentioned purposes of the invention, the present invention also provides a computer storage medium, comprising: at least one processor; a memory, wherein the memory stores a computer program that can be run on the processor, and when the processor executes the program, it performs any step of the method for determining the image code.

[0022] Compared with the prior art, the embodiments of the present invention have at least one of the following beneficial effects:

[0023] The present invention adopts a method for determining image codes. By dividing a page image into multiple grid sub-regions and extracting local features in each sub-region for matching, the detailed information in the page image can be captured more accurately. Compared with global image matching, this local-to-local matching method can reduce the mismatch problem caused by the overall similarity of the image but large differences in local details. When the position or style of an element in the page image changes, it is only necessary to re-match the sub-features of the grid where the element is located without reprocessing the entire image, thereby improving the efficiency and flexibility of determining image code generation. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 It is a schematic diagram of the workflow of the system for determining image codes in one embodiment of the present invention.

[0025] Figure 2 It is a schematic diagram of an initial page design draft image in one embodiment of the present invention.

[0026] Figure 3 It is a schematic diagram of obtaining a page image in one embodiment of the present invention.

[0027] Figure 4 It is a schematic diagram of the display of target code and target control in one embodiment of the present invention.

[0028] Figure 5 It is a schematic diagram of the steps of a method for determining an image code in one embodiment of the present invention.

[0029] Figure 6 It is a schematic diagram of steps before step S1 in one embodiment of the present invention.

[0030] Figure 7 It is a schematic diagram of the mapping relationship between component images and corresponding component codes in one embodiment of the present invention.

[0031] Figure 8 It is a schematic diagram of step S3' in an implementation of the present invention.

[0032] Fig. 9 It is a schematic diagram of step S3' in an implementation of the present invention.

[0033] Fig.10It is a schematic diagram of step S3 in one implementation of the present invention.

[0034] Fig.11 It is a schematic diagram of a mapping relationship stored in a preset knowledge base in a preferred embodiment of the present invention.

[0035] Fig.12 It is a schematic diagram of the steps of a method for determining an image code in one embodiment of the present invention.

[0036] Fig.13 A schematic structural diagram of an electronic device in one embodiment of the present invention. DETAILED DESCRIPTION

[0037] The present invention will be described in detail below in conjunction with the specific embodiments shown in the accompanying drawings. However, these embodiments do not limit the present invention, and any structural, methodological, or functional changes made by a person skilled in the art based on these embodiments are all within the scope of protection of the present invention.

[0038] It should be noted that the term "comprises" or any other variation thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device that includes a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In addition, the terms "first", "second", etc. are used for descriptive purposes only and cannot be understood as indicating or implying relative importance.

[0039] like Figure 1 As shown, an embodiment of the present invention provides a system 100 for determining an image code.

[0040] In one embodiment, the image code determination system 100 determines the target sub-features of the web page image according to a method for determining image codes, and generates corresponding target codes according to the similarity matching results of all target sub-features and candidate sub-features in the knowledge base, thereby eliminating the time cost of manually writing and adjusting codes and avoiding code errors caused by human errors. In addition, for web pages that need to be updated frequently or a large number of web pages with similar structures, the system can quickly generate corresponding codes, improving the response speed and the ability to adapt to changes.

[0041] In a specific embodiment, the method for determining the image code may be implemented with reference to any technical solution provided below.

[0042] The system 100 for determining an image code includes a client 11. The client 11 is used to determine a page image to be converted.

[0043] In one embodiment, the page image may be a design draft page image. In this embodiment, the initial page design draft is obtained and can be transmitted to the client 11 in the form of an image or a picture (such as a picture in a format such as PNG, JPEG, etc.).

[0044] In a specific embodiment, the initial page design draft may be created in professional design software and transmitted to the client 11 in the form of a screenshot.

[0045] like Figure 2 As shown, in a specific embodiment, the design draft page image is determined based on the Web (World Wide Web) terminal. Specifically, the initial design draft page image is obtained; based on the area selection box of the interactive graphical interface, drag and position operations are performed to determine that the local area selected by the selection box is the page image to be converted.

[0046] The initial design draft page image may be stored in a directory of a Web server, or obtained from an external server based on a URL (Uniform Resource Locator), and there is no specific limitation on this.

[0047] Use HTML (HyperText Markup Language) tags or CSS (Cascading Style Sheets) background images to display the initial design draft page image on the web page, ensuring that the design draft page image can be displayed in the original proportion or specified size so that users can clearly see the content. (Also called, the loading operation of the initial design draft page image)

[0048] Add a draggable element to the page and use it as a container for the selection box. You can define the color, width, and necessary styles (such as border style) of the border around the element to form a selection box effect. For example, you can set a rectangular black selection box. For details, see Figure 2 shown.

[0049] Use JavaScript to add dragging functionality, listen to mouse press, move, and release events, and update the position and size of the selection box. During the dragging process, dynamically adjust the position and size of the selection box according to the mouse position change, thereby determining the local area on the design draft. By adjusting the position and size of the selection box, the currently selected area is displayed to the user in real time. (Also, the box selection operation of the local area in the initial design draft page image)

[0050] Based on the position and size of the selection box, the corresponding image area is determined to be the design draft page image, such as Figure 3This process can be processed on the client 11 or on the server 11, and there is no specific limitation on this. (Alternatively, the generation operation of the design draft page image)

[0051] The image code determination system 100 includes a server end 12 .

[0052] In one embodiment, the server 12 is used to receive the page image transmitted from the client 11.

[0053] In one embodiment, the server 12 is used to divide the page image into grids.

[0054] It should be noted that in this embodiment, the grid division is a virtual grid division strategy, which logically divides the page image into multiple areas, but does not actually physically cut or divide the page image. The image data remains intact, but is treated as multiple independent parts during data processing.

[0055] In one embodiment, the server 12 is used to extract features of local page images in all grid sub-areas to obtain a number of target sub-features.

[0056] In a specific implementation, based on a pre-trained neural network model, feature prediction is performed on the local page image in the grid sub-region to obtain a number of target sub-features. For details, please refer to the following and will not be described in detail here.

[0057] In one embodiment, the server 12 is used to perform matching on a plurality of target sub-features based on a preset knowledge base to determine a target code that matches the page image.

[0058] In addition, the target code generated by the server 12 is transmitted to the display area of ​​the client 11 for editing or display.

[0059] like Figure 4 As shown, in one embodiment, the display area includes a first area and a second area; the first area is used to display the target code, and when it is detected that the target code in the first area is modified, the second area displays the corresponding modified target control.

[0060] The second area is a preview area that can be viewed. After the target code is edited in the first area, the target control in the second area will also change accordingly.

[0061] In addition, controls represent graphical interface elements that users can interact with. Controls allow users to input data, select options, operate objects, etc., and are the bridge for interaction with users in software applications. Controls can be divided into basic controls, such as buttons, text boxes, check boxes, labels, etc.; they can also be divided into list controls, such as list boxes, drop-down lists, etc. Of course, the present invention does not exclude other controls.

[0062] In one embodiment, the first area includes a save button, through which the final component code can be saved.

[0063] like Figure 5 As shown, an embodiment of the present invention provides a method for determining an image code.

[0064] The method for determining an image code is applied to a system for determining an image code.

[0065] In one embodiment, the system for determining the image code may be as follows: Figure 1 As configured as described above, the corresponding technical solution settings are referenced in the determination method of the present invention.

[0066] like Figure 5 As shown, an embodiment of the present invention provides a method for determining an image code, including the following steps.

[0067] Step S1, receiving a page image to be converted transmitted from a client;

[0068] Step S2, gridding the page image, and extracting features of the local page image in all grid sub-areas to obtain a number of target sub-features;

[0069] Step S3, based on a preset knowledge base, performing matching on a plurality of target sub-features to determine a target code matching the page image.

[0070] In this way, by dividing the page image into multiple grid sub-regions and extracting local features in each sub-region for matching, the detail information in the page image can be captured more accurately. Compared with global image matching, this local-to-local matching method can reduce the mismatch problem caused by the overall similarity of the image but large differences in local details. When the position or style of an element in the page image changes, it is only necessary to re-match the sub-features of the grid where the element is located without reprocessing the entire image, thereby improving the efficiency and flexibility of code generation.

[0071] The grid division in step S2 is to logically divide the page image into multiple sub-areas (i.e., grids), each of which represents a portion of the page image, but this division is virtual and does not involve actual modification or cutting of the image data. The specific description can be referred to above, and will not be repeated here.

[0072] Extracting features is the process of identifying and extracting information or attributes useful for a specific task or analysis from page image data. The features include at least one of shape, color, texture, edge and corner point.

[0073] In one embodiment, uniform grid division is adopted, that is, the page image or the designated area is equally divided into several parts in the horizontal and vertical directions.

[0074] In a specific embodiment, the page image is divided into nine equal parts, namely a 3x3 grid, forming a nine-square grid. In this way, each grid block can be processed independently or in parallel, which speeds up the processing speed and simplifies the processing process.

[0075] For example, if the size of the page image is height×width, then the size of each grid is height / 3×width / 3. A feature extraction operation is performed based on each grid area to obtain multiple target sub-features.

[0076] In one embodiment, feature extraction operations are performed on the local page images in all grid sub-regions to obtain a number of target sub-features. In this way, processing efficiency can be significantly improved through parallel processing.

[0077] The preset knowledge base is a collection of a large amount of knowledge, information or data that is pre-built and stored. The original intention of the knowledge base is to provide a centralized, easy-to-access and searchable information source so that users can quickly find the information or solutions they need.

[0078] In one embodiment, a historical page image is determined based on historical data, wherein the historical page image includes a plurality of component images, and the component image includes at least one of a basic component, a form component, a data display component, a notification component, and a navigation component.

[0079] In a specific embodiment, the historical page image refers to a page snapshot or page screenshot presented based on historical data, which includes all visual elements on the page. The component image is a portion of the historical page image that can be individually identified and processed.

[0080] For ease of processing, the collected component images can be classified according to function or type, and component images of the same type can be stored in the same folder. For example, the file names are classified into six categories, namely basic, form, data, notice, navigation, and others. The file naming formats are basic_***.png, form_***.png, data_***.png, notice_***.png, navigation_***.png, and others_***.png. The "***" in the file name can be replaced by a numerical sequence to distinguish different page images and does not have a special meaning, such as: basic_1.png / basic_2.png / form_1.png.

[0081] The component images are put into the collected_images folder, and the corresponding code files are saved in text format with the same name. The codes are put into the code_texts folder for subsequent association processing.

[0082] In addition, in order to simplify the complexity of image processing and improve the speed of image processing, the color component images collected in the collected_images folder can be batch converted into grayscale images to simplify the subsequent feature extraction process.

[0083] In order to further save time for subsequent component image feature extraction, a pre-trained neural network model may be used to extract features from the collected historical page images and / or component images to determine a number of corresponding candidate sub-features.

[0084] In one embodiment, it is determined whether each historical page image or component image is a grayscale image; if not, the historical page image or component image is converted into a corresponding grayscale image. In this way, by removing color information and retaining only brightness information, subsequent feature extraction operations are simplified.

[0085] In one embodiment, the grayscale image is gridded to determine a plurality of grid sub-region images.

[0086] In one embodiment, based on a pre-trained convolutional neural network model, the features of each grid sub-region image are extracted to obtain a number of candidate sub-features, thereby improving the efficiency of feature extraction, achieving a higher degree of automation and higher accuracy.

[0087] The Convolutional Neural Network (CNN) is a deep learning model that automatically extracts features of input data by simulating the hierarchical processing mechanism of visual information by the human brain. Specifically, the convolution kernel in the convolution layer slides on the input data, processing only a local area of ​​the data each time, thereby realizing local feature extraction of data such as images.

[0088] In a specific embodiment, the convolutional neural network model is ResNet (Residual Neural Network).

[0089] ResNet is a deep neural network structure that solves the gradient vanishing and gradient exploding problems in the deep network training process by introducing residual connections, effectively deepening the network and improving model performance.

[0090] like Figure 6 As shown, in one embodiment, before the pre-trained convolutional neural network model, the method further includes:

[0091] Step P1, update the number of channels of the initial convolutional neural network model to a single channel, and adjust the mean and variance of the single channel;

[0092] Step P2, training the initial convolutional neural network model based on the training data to obtain a trained convolutional neural network model.

[0093] In this way, by modifying the number of channels, mean and variance of the network model, the model can be adapted to grayscale images with a single channel, which can improve the model's feature extraction ability and generalization performance when processing grayscale images, and ensure that the model can learn key information from grayscale images more effectively.

[0094] Specifically, installing the PyTorch and torchvision libraries, loading the ResNet18 model, changing the first layer of the model from multi-channel (such as RGB three-channel) to single-channel, and adjusting normalization parameters such as mean and variance can make the model more focused on the brightness information of the grayscale image rather than the color information, thereby more accurately extracting image features and improving the accuracy and efficiency of the model on grayscale image tasks.

[0095] In one embodiment, a grid index is determined based on the position of the grid sub-region after grid division; and a mapping relationship between the grid index and the corresponding candidate sub-feature is constructed.

[0096] In a specific embodiment, the target sub-feature can be converted into a corresponding feature vector; the candidate sub-feature can also be converted into a corresponding feature vector.

[0097] Specifically, taking the nine-grid grid as an example, the historical page image is evenly divided into nine sub-regions, and the position number of the sub-region is used as the grid index. The sub-features corresponding to each sub-region page image are associated with the grid index through code, and saved in a JSON file to form a feature index file. In other words, the feature index file is used to store the mapping relationship between feature vectors and grid indexes.

[0098] like Figure 7 As shown, in one embodiment, a mapping relationship between a component image and a corresponding component code is established, so that the corresponding target code can be quickly determined based on feature positioning.

[0099] Specifically, the feature index in the knowledge base is loaded from the stored JSON file, wherein the feature index includes a feature vector, a grid position, an index and a code.

[0100] In one embodiment, step S3 may specifically include the following steps.

[0101] Step S3', based on the cascade matching method, determine the candidate sub-features matching the target sub-features in the preset knowledge base, and determine the corresponding target code according to the candidate sub-features.

[0102] In this way, by adopting the cascade matching method, a large number of irrelevant candidate objects can be quickly filtered out in the early stage, the matching scope can be gradually reduced, and the matching speed and accuracy can be improved.

[0103] The cascade matching is a multi-stage matching process that combines multiple matching steps in a certain order to form a hierarchical matching process. Each step will filter and select the data to make the final recommendation or matching result more in line with the user's needs or expectations.

[0104] This method decomposes the complex matching task into multiple simple subtasks, each of which is screened for specific features or attributes, and gradually narrows the candidate set to eventually determine the target set.

[0105] like Figure 8 As shown, in a specific embodiment, step S3' may specifically include the following steps.

[0106] Step S311, determining a first candidate feature set based on the similarity between the first target sub-feature and the candidate sub-features in the preset knowledge base;

[0107] Step S312, determining a second candidate feature set based on the similarity between the second target sub-feature and the candidate sub-features in the first candidate feature set;

[0108] Step S313: based on the second candidate feature set, iteratively determine the candidate sub-features that match other target sub-features until the similarities of all target sub-features meet the set conditions, and determine the final candidate sub-features.

[0109] In this way, by gradually narrowing the candidate range and iteratively matching multiple target sub-features, candidate sub-features that do not meet the conditions are gradually eliminated, thereby improving the matching accuracy and matching efficiency.

[0110] The similarity is a numerical indicator that measures the degree of similarity between two or more objects. In the present invention, the similarity refers to the similarity between the target sub-feature and the candidate sub-feature in the knowledge base. The present invention does not impose any specific restrictions on the selection of the similarity calculation method.

[0111] The similarity calculation method includes at least one of cosine similarity and Euclidean distance. For example, the similarity can be measured by calculating the Euclidean distance between two sub-features. If the distance is smaller, the similarity is higher; conversely, if the distance is larger, the similarity is lower. In the above embodiment, it is a serial process based on matching of several target sub-features in sequence.

[0112] like Fig. 9 As shown, in one embodiment, step S3' may further specifically include the following steps.

[0113] Step S321, obtaining all data blocks in a preset knowledge base, each data block pointing to the grid division information of the corresponding candidate page image, the grid division information including a number of candidate sub-features after the candidate page image is grid-divided;

[0114] Step S322, constructing a corresponding index for each data block, and adding the corresponding candidate sub-features to the index to form a plurality of feature indexes;

[0115] Step S323: Based on the feature index, perform a similarity search on the target sub-feature to determine a candidate sub-feature whose similarity meets a preset condition.

[0116] In this way, by constructing a feature index, it can be ensured that the candidate sub-features of each data block can be matched independently, thereby improving the matching speed and accuracy.

[0117] The index is a complex data structure for storing, organizing and quickly retrieving multiple sub-features to support efficient similarity search operations.

[0118] In one embodiment, the index is a FAISS (Facebook AI Similarity Search) index.

[0119] FAISS index is a query method based on vector similarity. It maps vector data into an efficient retrieval structure through encoding and indexing technology. It can be applied to real-time search of large-scale data sets and focuses more on vector similarity calculation and fast retrieval. In order to improve the efficiency and accuracy of similarity matching, FAISS index can be used to screen sub-grids one by one, gradually filter out non-conforming matches, and reduce the number of matches.

[0120] Specifically, in the preset knowledge base, the candidate page images are stored as the result of grid division (including grid index and candidate sub-features), and multiple grid blocks (i.e., data blocks) can be generated. The database stores the grid division information about the candidate page images, and determines the sub-features of each sub-region according to the grid division information to obtain several candidate sub-features; the candidate sub-features are added to the similarity search index so as to be efficiently compared and searched with the target sub-features.

[0121] It should be noted that in the process of matching each target sub-feature with the candidate sub-features in the knowledge base, flexible screening is performed based on the set similarity threshold, and the scope and intensity of the screening can be controlled.

[0122] In one embodiment, the FAISS index may also be applied in the process of matching the target sub-feature with the candidate sub-features in the preset knowledge base. In other words, steps S321 to S323 may be arranged before step S311, or may be arranged before step S312.

[0123] In addition to the cascade matching method, in order to fully consider the features in each grid sub-area and prevent a sub-area from being ignored due to feature matching failure in the cascade matching process, the "full matching and intersection screening method" can also be used.

[0124] like Fig.10 As shown, in one embodiment, step S3 may specifically include the following steps.

[0125] Step S331, based on the similarity between each target sub-feature and the candidate sub-features in the preset knowledge base, determine a corresponding number of candidate feature sets;

[0126] Step S332, determining the corresponding target code according to the intersection of all candidate feature sets.

[0127] In this way, the comprehensive influence of all grid features can be considered more comprehensively, and the intersection can be calculated to ensure that the final matching candidate features have a high matching degree in all data blocks in the knowledge base.

[0128] In one embodiment, the candidate sub-codes corresponding to all candidate sub-features in the intersection result are merged to form the target code.

[0129] In one embodiment, operations related to the creation and use of FAISS indexes may be added to step S331 to step S332, that is, step S321 to step S323 may be set before step S331. In other words, before matching the candidate sub-features of each target sub-feature, a FAISS index may be created for the data block in the knowledge base, and a matching operation may be performed based on the FAISS index.

[0130] like Fig.11 As shown, based on image acquisition and image processing, each data block in the preset knowledge base (i.e., corresponding candidate page image) can generate nine index records for storing the mapping relationship between the candidate page image, the candidate sub-features, the grid index and the corresponding candidate image code. Fig.11 In , each data block includes 9 grid indexes (Grid: 0-8), and the corresponding candidate page images are the same, and the corresponding candidate image codes are also the same.

[0131] After the server matches and obtains the final target code, it is transmitted to the client and displayed to the user through the client. At the same time, the user is provided with the function of editing and modifying, so as to allow the user to adjust and optimize the target code in real time according to actual needs, thus enhancing the user experience.

[0132] like Fig.12 As shown, in one embodiment, the method for determining an image code according to the present invention may further include:

[0133] Step S4, creating a web application, the web application including a first route and a second route, the first route is used to render a page including the target code editing area, and the second route is used to save the editing operation of the target code;

[0134] Step S5: Based on the first route and the second route, the target code is transmitted to the client for display and editing.

[0135] In this way, by providing the user with a page with an editable area of ​​the target code, the user can make adjustments according to actual needs, thereby enhancing the user experience.

[0136] In one embodiment, the web application is a Flask application.

[0137] The Flask application is a lightweight web application framework written in Python that allows for rapid design and development of web applications. In a Flask application, first create an instance of the Flask application, which is the core of the Flask application and is responsible for processing web requests, managing routes, rendering templates, etc.

[0138] In one embodiment, the first route is an index route. The index route refers to a route corresponding to a default view or operation of a resource or controller, which can be associated with the "index" part in the URL path and is used to represent the entry of the default view or operation of the resource list.

[0139] The index route is used to render the initial page or homepage corresponding to the web application, such as the front-end page. In the Flask application, when the user accesses this route, the server returns an HTML page that contains an edit area, a reserved area, and a save button. In the HTML page, the edit area allows the user to enter or modify the code, the preview area displays the effect of the code in real time, and the save button is used to submit the code for saving.

[0140] In one embodiment, the second route is a save_code route. The save_code route is a custom route in a specific application or framework, which can be responsible for receiving a request for data sent by the front end and executing corresponding request logic at the back end.

[0141] The save_code route is used to process data submitted by the user. When the user modifies the code in the edit area and clicks the Save button, the form data can be sent to this route via a POST request. The server will receive the code data and save it to a specified location (for example, a database or file system) for subsequent use or viewing. At the same time, a response can be returned to the user, which may be a redirect to another page (such as the index route) or a JSON object containing the results of the operation.

[0142] like Fig.13 As shown, an embodiment of the present invention provides an electronic device 200.

[0143] The electronic device 200 may specifically be a computer device, and the computer device may be a terminal device or a server.

[0144] The electronic device 200 includes at least one processor. The method for determining an image code provided by the present invention can be applied to the processor or implemented by the processor. The processor can be a central processing unit (CPU) 21.

[0145] The electronic device 200 includes a memory. The memory is used to store various types of data to support the operation of the electronic device 200. Examples of such data include any computer program for operating on a computer device. The memory may be a read-only memory (ROM) 22, a random access memory (RAM) 23, or other storage part 28. The storage part 28 may be located inside the electronic device 200 or outside the electronic device 200.

[0146] In one embodiment, when the processor executes the computer program stored in the memory, the steps of the method for determining the image code in any technical solution of the present invention are performed.

[0147] In one embodiment, the electronic device 200 includes a central processing unit 21, which can perform various appropriate actions and processes according to the program stored in the read-only memory 22 or the program loaded from the storage part 28 into the random access memory 23. Various programs and data required for system operation are also stored in the random access memory 23. The central processing unit 21, the read-only memory 22 and the random access memory 23 are connected to each other through a bus 24. An input / output interface (Input / Output interface, i.e., I / O interface) 25 is also connected to the bus 24.

[0148] The following components are connected to the input / output interface 25: an input section 26 including a keyboard, a mouse, etc.; an output section 27 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 28 including a hard disk, etc.; and a communication section 29 including a network interface card such as a LAN card, a modem, etc. The communication section 29 performs communication processing via a network such as the Internet. A drive 210 is also connected to the input / output interface 25 as needed. A removable medium 211, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 210 as needed, so that a computer program read therefrom is installed into the storage section 28 as needed.

[0149] One embodiment of the present invention provides a computer-readable storage medium.

[0150] In one embodiment, a computer readable storage medium stores a computer program executed by the processor mentioned above, or a method for determining an image code in any of the technical solutions described below.

[0151] When the processor executes the computer program, it can execute the description of the method for determining the image code in any of the technical solutions below, so it will not be repeated here. In addition, the description of the beneficial effects of using the same method will not be repeated.

[0152] The computer-readable storage medium may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0153] In summary, the method, system, device and storage medium for determining image codes provided by the present invention can more accurately capture the detail information in the page image by dividing the page image into multiple grid sub-regions and extracting local features in each sub-region for matching. Compared with global image matching, this local-to-local matching method can reduce the problem of mismatching caused by overall similarity of images but large differences in local details. When the position or style of an element in the page image changes, it is only necessary to re-match the sub-features of the grid where the element is located without reprocessing the entire image, thereby improving the efficiency and flexibility of code generation.

[0154] It should be understood that although this specification is described according to implementation modes, not every implementation mode contains only one independent technical solution. This description of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each implementation mode may also be appropriately combined to form other implementation modes that can be understood by those skilled in the art.

[0155] The series of detailed descriptions listed above are only specific descriptions of feasible implementation methods of the present invention. They are not intended to limit the scope of protection of the present invention. Any equivalent implementation methods or changes that do not deviate from the technical spirit of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for determining an image code, It is characterized in that include: Receiving a page image to be converted transmitted from a client; Dividing the page image into grids, and extracting features of the local page image in all grid sub-regions to obtain a number of target sub-features; Based on a preset knowledge base, matching is performed on a plurality of target sub-features to determine a target code that matches the page image.

2. The method for determining an image code according to claim 1, It is characterized in that The performing matching on a plurality of target sub-features to determine a target code matching the page image includes: Based on the cascade matching method, candidate sub-features matching the target sub-features in the preset knowledge base are determined, and the corresponding target codes are determined according to the candidate sub-features.

3. The method for determining an image code according to claim 2, It is characterized in that The method of determining a candidate sub-feature matching the target sub-feature in a preset knowledge base based on the cascade matching method includes: Determine a first candidate feature set based on the similarity between the first target sub-feature and the candidate sub-features in the preset knowledge base; Determining a second candidate feature set based on similarities between the second target sub-feature and the candidate sub-features in the first candidate feature set; Based on the second candidate feature set, candidate sub-features matching other target sub-features are iteratively determined until the similarities of all target sub-features meet the set conditions, and the final candidate sub-features are determined.

4. The method for determining an image code according to claim 2, It is characterized in that The method of determining a candidate sub-feature matching the target sub-feature in a preset knowledge base based on the cascade matching method includes: Acquire all data blocks in a preset knowledge base, each data block pointing to grid division information of a corresponding candidate page image, the grid division information including a plurality of candidate sub-features after grid division of the candidate page image; Build a corresponding index for each data block, and add the corresponding candidate sub-features to the index to form several feature indexes; Based on the feature index, a similarity search is performed on the target sub-feature to determine candidate sub-features whose similarity meets a preset condition.

5. The method for determining an image code according to claim 1, It is characterized in that The performing matching on a plurality of target sub-features to determine a target code matching the page image includes: Based on the similarity between each target sub-feature and the candidate sub-features in the preset knowledge base, determine a corresponding number of candidate feature sets; According to the intersection of all candidate feature sets, the corresponding target code is determined.

6. The method for determining an image code according to claim 5, It is characterized in that Determining the corresponding target code according to the intersection of all candidate feature sets includes: The candidate sub-codes corresponding to all the candidate sub-features in the intersection result are merged to form the target code.

7. The method for determining an image code according to claim 1, It is characterized in that After determining the target code matching the page image, the method further comprises: Creating a web application, the web application comprising a first route and a second route, the first route being used to render a page including the target code editing area, and the second route being used to save the editing operation on the target code; Based on the first route and the second route, the target code is transmitted to a client for display and editing.

8. The method for determining an image code according to claim 1, It is characterized in that Before dividing the page image into grids, the method further includes: Determining whether the page image is a grayscale image; If not, converting the page image into a corresponding grayscale image; The features of the local page images in all grid areas are extracted to obtain several target sub-features, including: Based on the pre-trained convolutional neural network model, the features of each local page image are extracted to obtain several target sub-features.

9. The method for determining an image code according to claim 8, It is characterized in that Before the pre-trained convolutional neural network model, the method further includes: Update the number of channels of the initial convolutional neural network model to a single channel, and adjust the mean and variance of the single channel; The initial convolutional neural network model is trained based on the training data to obtain a trained convolutional neural network model.

10. A system for determining an image code, It is characterized in that include: The client is used to determine the page image to be converted; The server side is used to receive the page image transmitted from the client, and to grid the page image, extract the features of the local page image in all grid sub-areas, and obtain a number of target sub-features; and to perform matching on the several target sub-features based on a preset knowledge base to determine a target code that matches the page image.

11. The system for determining an image code according to claim 10, It is characterized in that The target code is transmitted to the display area of ​​the client for editing and / or display, and the display area includes a first area and a second area; the first area is used to display the target code, and when it is detected that the target code in the first area is modified, the second area displays the corresponding modified target control.

12. A refrigeration device, include: at least one processor; A memory storing a computer program executable on the processor, wherein the processor executes the steps of the method for determining an image code according to any one of claims 1 to 9 when executing the program.

13. A computer storage medium, include: at least one processor; A memory storing a computer program executable on the processor, wherein the processor executes the steps of the method for determining an image code according to any one of claims 1 to 9 when executing the program.

Images