Component searching method and device, equipment, storage medium and program product

Abstract

The invention relates to the technical field of computers, and discloses a component searching method and device, equipment, a storage medium and a program product, and the method comprises the steps: obtaining a to-be-searched target component picture; performing image processing on the target component picture through a picture search component model, and determining a target image vector corresponding to the target component picture; searching a target text vector closest to the target image vector based on the mapping relation through an image search component model; and determining a component corresponding to the target text vector as a component search result corresponding to the target component picture. According to the technical scheme, the effect of searching the target component from the component library through the component image is achieved, the convenience in the component searching and multiplexing process is greatly improved, the component searching difficulty is reduced, the development period of a data service scene is shortened, and the development efficiency of the data service scene is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a component search method, device, equipment, storage medium, and program product. Background Art

[0002] When developers develop data business scenarios, they usually carefully observe the design drawings of the data business scenarios to determine the data components required for the data business scenarios so that they can select available components from the business component library for application. However, with the continuous expansion of data business, the number of data components required for data business scenarios is increasing, and the number of components in the business component library will continue to increase. It takes a lot of time to find components one by one, and the design drawings are presented in the form of images. Developers cannot search by component name or related description text, which increases the difficulty of finding components and affects the development efficiency of data business scenarios. Summary of the invention

[0003] In view of this, the present disclosure provides a component search method, apparatus, device, storage medium and program product to solve the problem that it is difficult to search for components through images.

[0004] In a first aspect, the present disclosure provides a component search method, including: obtaining a target component image to be searched; extracting component information corresponding to the target component image; inputting the component information into a pre-trained image search component model, and outputting component search results corresponding to the target component image through the image search component model; wherein the image search component model is trained based on component image samples and component description texts corresponding to the component image samples.

[0005] In a second aspect, the present disclosure provides a component search device, including: an acquisition module for acquiring a target component image to be searched; an information extraction module for extracting component information corresponding to the target component image; a search module for inputting the component information into a pre-trained image search component model, and outputting component search results corresponding to the target component image through the image search component model; wherein the image search component model is trained based on component image samples and component description texts corresponding to the component image samples.

[0006] In a third aspect, the present disclosure provides a computer device, comprising: a memory and a processor, the memory and the processor are communicatively connected to each other, computer instructions are stored in the memory, and the processor executes the component search method of the first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0007] In a fourth aspect, the present disclosure provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the component search method of the first aspect or any corresponding embodiment thereof.

[0008] In a fifth aspect, the present disclosure provides a computer program product, including computer instructions, where the computer instructions are used to enable a computer to execute the component search method of the first aspect or any corresponding embodiment thereof.

[0009] The component search method, apparatus, device, storage medium and program product provided by the present disclosure use component image samples and component description texts corresponding to the component image samples to train the image search component model, and deploy the image search component model to the server. Thus, when receiving the target component image input by the user, the component information in the target component image can be extracted, and the extracted component information is input into the image search component model, so that the image search component model can perform component search according to the target component image and obtain corresponding component search results. Thereby, the effect of searching the target component from the component library through the component image is achieved, which greatly improves the convenience in the component search and reuse process. There is no need for developers to manually browse the huge business component library to find similar components, which reduces the difficulty of component search, thereby shortening the development cycle of data business scenarios and improving the development efficiency of data business scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the specific embodiments of the present disclosure or the technical solutions in the prior art, the drawings required for use in the specific embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0011] Figure 1 is a schematic diagram of an indicator card component according to an embodiment of the present disclosure;

[0012] Figure 2 is a schematic diagram of component search according to an embodiment of the present disclosure;

[0013] Figure 3 is a block diagram of an implementation of component search according to an embodiment of the present disclosure;

[0014] Figure 4 is a schematic diagram of an automated process of component search according to an embodiment of the present disclosure;

[0015] Figure 5 is a flowchart of a component search method according to an embodiment of the present disclosure;

[0016] Figure 6is a flowchart of a training method for a graph search component model according to an embodiment of the present disclosure;

[0017] Figure 7 is a flowchart of another training method of a graph search component model according to an embodiment of the present disclosure;

[0018] Figure 8 is a flowchart of another method for training a graph search component model according to an embodiment of the present disclosure;

[0019] Fig. 9 is a schematic diagram of a model architecture of a graph search component according to an embodiment of the present disclosure;

[0020] Fig.10 is a structural block diagram of a component search device according to an embodiment of the present disclosure;

[0021] Fig.11 It is a schematic diagram of the hardware structure of the computer device of the embodiment of the present disclosure. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical solution and advantages of the embodiments of the present disclosure clearer, the technical solution in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, rather than all the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present disclosure.

[0023] It is understandable that before using the technical solutions disclosed in the embodiments of the present disclosure, the types, scope of use, usage scenarios, etc. of the personal information involved in the present disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0024] For example, in response to receiving an active request from a user, a prompt message is sent to the user to clearly prompt the user that the operation requested to be performed will require obtaining and using the user's personal information. Thus, the user can autonomously choose whether to provide personal information to software or hardware such as an electronic device, application, server, or storage medium that performs the operation of the technical solution of the present disclosure according to the prompt message.

[0025] As an optional but non-limiting implementation, in response to receiving an active request from the user, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. In addition, the pop-up window may also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0026] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation on the implementation of the present disclosure. Other methods that meet the relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0027] It is understandable that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and relevant provisions.

[0028] The application development of data business scenarios has a corresponding business component library. This business component library follows the standardized data component design system and can establish data interaction specifications from the perspectives of presentation, data presentation, data visualization, and data interaction. When the developers of the data department develop for data business scenarios, they will first obtain the design drawings provided by the designer for the data business scenarios, and carefully observe the components involved in the design drawings (such as Figure 1 The indicator card component shown in the figure) is used to search for reusable components in the business component library. If a suitable component is found, the developer will directly apply it.

[0029] However, due to the large number of components involved in the design, searching one by one would take a lot of time. In addition, as data services continue to expand, the number of data components in the business component library is also increasing, which makes the process of finding similar components more complicated. Since the design is presented in the form of an image, developers cannot search by component name or related description text, which further increases the difficulty of searching.

[0030] Based on this, the core of the technical solution disclosed in this disclosure is to train the image search component model, and support the search function based on component images through the image search component model, so as to recommend multiple component names that are the same or similar to the component images uploaded by the user and the components corresponding to each component name, such as Figure 2 Component search results are shown.

[0031] like Figure 3As shown in the implementation block diagram, the infrastructure / dependency layer is used to provide the dependency environment for the training of the graph search component model, such as text vector model, screenshot service, multimodal large language model interface, vector database, etc. At the same time, it provides a database for data storage. The basic service layer provides specific dependencies for the infrastructure / dependency layer, including component service, screenshot service, large model service, vectorization service and model service. Among them, the component service is used to provide service implementations for obtaining components, obtaining initial component attribute data, calculating incremental components, storing component description data, and rendering components; the screenshot service is used to provide service implementations for downloading component description data, screenshots, and storage; the large model service is used to provide service implementations for generating component attribute data and generating component description text; the vectorization service is used to provide service implementations for component description text conversion, storage in vector database, obtaining vectors from vector database, and searching for the top k similar vectors; the model service is used to provide service implementations for downloading screenshots to local, downloading component description text vectors to local, data enhancement and expansion, dividing data sets, model training, model testing, model ranking, and updating graph search components. The service layer is used to provide search services for the image search component, including providing the search interface of the image search component and deploying the image search component model. The application layer is used to provide image search component applications in the business component library.

[0032] In order to support the above functions, a rich component image dataset is required for training. However, manually collecting these datasets is time-consuming and laborious. The disclosed technical solution automates the entire process of dataset collection, model training, model testing, and service deployment, greatly improving R&D efficiency. This automation can not only significantly shorten the development cycle, but also reduce human errors and improve the quality and consistency of the dataset, thereby laying a solid foundation for subsequent model optimization and performance improvement.

[0033] The automation process is as follows: Figure 4As shown in the figure, when developing components in the business component library, when the developer submits the merge main branch operation, the execution of the entire graph search component will be automatically triggered. First, multiple sets of rich component property (props) data are generated to provide diverse training data for the training of the graph search component model; second, the components are rendered using the expanded component property data; third, the rendered components are screenshotted through the screenshot service, and the component screenshots are saved in the database; then, based on the component screenshots and component names, relevant component description texts are generated for each type of component, and the component description texts are converted into text vectors using the text vector model, and the text vectors are used as the unique identifier of each type of component; finally, the graph search component model is automatically trained and tested using the mapping relationship between the text vector, the image vector corresponding to the component screenshot, and the text vector and the image vector. After the training of the graph search component model is completed, the deployment of the graph search component model is automatically triggered to update the graph search component service to ensure that the graph search component service always uses the latest graph search component model.

[0034] In the process of model training, using description text related to component style and function to represent components will ensure that the model can more accurately capture and understand the characteristics of components, thereby improving the accuracy and efficiency of component search based on component images. By optimizing data structure in a targeted manner, the search needs of component images can be better met, and the effectiveness of recommending similar components can be promoted.

[0035] According to an embodiment of the present disclosure, a component search method embodiment is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0036] In this embodiment, a component search method is provided, which can be used in computer devices, such as computers, mobile phones, tablet computers, etc. Figure 5 is a flowchart of a component search method according to an embodiment of the present disclosure. Figure 5 As shown, the process includes the following steps:

[0037] Step S101, obtaining a target component image to be searched.

[0038] The target component image is the image that needs to be queried for the component, that is, the component image provided by the user based on the actual usage scenario. Specifically, the computer device provides an interactive interface, and the user can upload the target component image to the computer device through the interactive interface. Among them, the target component image can be captured by the user calling the screenshot service and uploaded to the computer device; it can also be selected by the user from the local gallery and uploaded to the computer device; it can also be downloaded from the online and uploaded to the computer device by the user. The method for obtaining the target component image is not specifically limited here, and those skilled in the art can determine it according to actual needs.

[0039] Step S102, performing image processing on the target component image through the image search component model to determine the target image vector corresponding to the target component image.

[0040] The target image vector is a feature vector representation of the component image. The component image of the target component to be searched is input into the image search component model, and the image search component model is used to extract features of the component image to obtain the target image vector corresponding to the component image.

[0041] Step S103, searching the target text vector closest to the target image vector based on the mapping relationship through the image search component model.

[0042] Among them, the image search component model is trained based on component image samples and the component description text corresponding to the component image samples.

[0043] The image search component model is a component search model that is pre-trained and deployed in a computer device. The image search component model can be a large language model, a deep learning model, a machine learning model, a neural network model, etc. The model architecture of the image search component model is not specifically limited here, as long as the component search function can be realized.

[0044] According to the mapping relationship between the image vector and the text vector, one or more text vectors associated with the target image vector are searched in the vector database. The vector distance between the target image vector and each text vector is calculated by using the vector distance calculation method, and the vector distances between each text vector are compared to select the text vector with the closest vector distance, and the text vector with the closest distance is determined as the target text vector.

[0045] Step S104, determining the component corresponding to the target text vector as the component search result corresponding to the target component image.

[0046] The text vector is generated based on the component description text, which carries component related information, such as component name, component style, etc. When the target text vector corresponding to the target image vector is determined, the component related information is extracted from the target text vector, and the component corresponding to the target text vector is determined based on the component related information, and the component is determined as the component search result corresponding to the component image of the target component to be searched.

[0047] The component search method provided by the present disclosure uses component image samples and component description texts corresponding to the component image samples to train an image search component model, and deploys the image search component model to the server. Thus, when receiving a target component image input by a user, the component information in the target component image is extracted, and the extracted component information is input into the image search component model, so that the image search component model can perform component search according to the target component image and obtain corresponding component search results. Thus, the effect of searching for target components from the component library through component images is achieved, which greatly improves the convenience in the component search and reuse process. Developers do not need to manually browse the huge business component library to find similar components, which reduces the difficulty of component search, thereby shortening the development cycle of data business scenarios and improving the development efficiency of data business scenarios.

[0048] In this embodiment, a method for training a graph search component model is provided, so that after the training of the graph search component model is completed, it is deployed to a computer device for component search. Figure 6 is a flowchart of a training method for a graph search component model according to an embodiment of the present disclosure, such as Figure 6 As shown, the process includes the following steps:

[0049] Step S201, obtaining component attribute data corresponding to the component sample.

[0050] Component attributes are objects used to transfer data and configurations between components, allowing parent components to pass data to child components, thereby achieving component reuse and flexibility. Component attribute data refers to various characteristics and information related to component samples, which describe aspects such as component characteristics, component behavior, and component configuration. Specifically, component attribute data may include basic attributes such as component name, type, and identifier, behavioral attributes such as interaction mode and response events, configuration attributes such as parameter settings and option selections, and status attributes such as whether to enable or not. Of course, component attribute data may also include other attributes, which are not specifically limited here.

[0051] Specifically, for different model frameworks, the corresponding component attribute data is not the same. Different model frameworks have corresponding component definitions or component documents, which record detailed information of component attributes, such as name, data type, default value, identifier, etc. Component attribute data can be obtained by querying component definitions or component documents, and the obtained component attribute data is stored in the corresponding database.

[0052] Specifically, the model framework also provides specific methods or functions to obtain component attribute data. According to the implementation mode of the component, the specific method of the component is called to access the component's attributes, obtain the component attribute data, and store the obtained component attribute data in the corresponding database. Of course, component attribute data can also be obtained by other methods, which are not specifically limited here.

[0053] Step S202, performing component rendering according to the component attribute data to obtain a rendering component corresponding to the component attribute data.

[0054] The dependent environment of the rendering service is deployed in the computer device. When the component is rendered according to the component attribute data, the rendering task for the component is constructed through the rendering service. The component attribute data is downloaded from the database, and the component attribute data is bound to its corresponding component. Then, the component attribute data is injected into the rendering task to perform the component rendering operation, and the rendering component corresponding to the component attribute data is generated in the page.

[0055] In a specific example, an interactive component (React component) for receiving component attribute data is pre-set, and the content of the component attribute data is displayed through the interactive component. A file that can directly use the interactive component is used for component rendering, for example, an MDX file that can achieve seamless integration of content creation and component rendering, and the dynamic component is easily inserted, the component attribute data is transmitted, and the interactive content of the component is created.

[0056] Specifically, the acquired component attribute data is stored in an array, the component attribute data is imported into the MDX file, and the component attribute data in the array is traversed to render each set of component attribute data. Each set of component attribute data is bound to its corresponding component, and at the same time, it is ensured that each component has a unique identifier, thereby ensuring the efficiency of the rendering process and the correctness of the component.

[0057] Step S203, taking a screenshot of the rendering component to generate a component screenshot.

[0058] A page loading service and a screenshot service are deployed in a computer device. The web page displaying the rendering component is loaded through the page loading service. The rendering component displayed in the web page is captured through the screenshot service to obtain a component screenshot.

[0059] Specifically, a page loading service is called through a web page interface, the web page is loaded and rendered on the server side using the page loading service, and the rendering component is displayed in the web page. A screenshot service deployed in a computer device is called to take a screenshot of the rendering component in the web page to obtain a component screenshot of the rendering component.

[0060] Step S204: Generate component description text based on the component screenshot and component information of the rendered component.

[0061] Component information is used to represent the characteristic information of a component, such as component name, component style, component type, component function, etc. Component description text is a description of the component from the style and function level. The component description text is used to uniquely identify the effective information of the component so that the keywords in components with similar styles and functions are consistent.

[0062] Bind the component screenshot corresponding to the rendered component to the component information, perform image analysis on the component screenshot, extract relevant visual features in the component screenshot, such as component shape, color, layout, etc., and use image recognition technology to identify the component type in the component screenshot, such as buttons, text boxes, lists, etc. At the same time, extract the relevant properties and data of the component from the component information, such as component name, component function, component status, etc. Combine the visual features extracted from the component screenshot, component type, and component information of the rendered component, and use natural language processing technology to generate component description text. For example, this is a [component type] component with [function description] function, currently in [state description] state; the component name is [component name], used for [purpose description], and its color is [color description], etc.

[0063] Step S205: training the image search component model based on the component description text and the component screenshot.

[0064] Bind the component description text with its corresponding component screenshot, take the component screenshot and component description text as input, and take the component name and its corresponding component as output, so that the image search component model can retrieve the corresponding multiple components based on the component image. Specifically, extract visual features from the component screenshot, such as using a convolutional neural network (CNN) to extract image features of the component screenshot. Use a bag of words model, TF-IDF, or a pre-trained language model to convert the component description text into a vector representation, and use the extracted image features and vector representation to train the image search component model. Among them, the image search component model can select a model architecture suitable for image data and text data, such as a multimodal neural network.

[0065] The component search result is one or more components similar to the target component obtained by searching for the target component. During the training process, a loss function can be used to measure the difference between the component prediction results of the image search component model and the actual components, and optimized through the back propagation algorithm. After completing the training of the image search component model, the image search component model is deployed on the server. When the component image of the target component to be searched is received, the component image is input into the trained image search component model, the component description text corresponding to the component image is obtained, and the image search component model is used to predict the component based on the component description text, and the components that are the same or similar to the target component are output to obtain the component search result.

[0066] The training method of the image search component model provided in this embodiment obtains component attribute data, renders the component attribute data to obtain the corresponding rendering component, takes a screenshot of the rendering component to obtain a component screenshot, generates a component description text using the component screenshot and component information, and trains the image search component model through the component description text and the component screenshot. After completing the training of the image search component model, the image search component model is deployed to the server to perform component search based on the component image to obtain the corresponding component search result. Thus, the entire process from component attribute data set construction, image search component model training to image search component service deployment can be automated, achieving the effect of searching for target components from the component library through component images, reducing the difficulty of component search, and improving the development efficiency of data business scenarios.

[0067] In some optional implementations, the component attribute data includes first component attribute data and second component attribute data, wherein the first component attribute data is attribute data possessed by the component, and the second component attribute data is one or more sets of attribute data obtained by expanding the first component attribute data. Figure 7 As shown, the above step S201 may include:

[0068] Step S2011, obtaining first component attribute data of the component sample and attribute features corresponding to the first component attribute data.

[0069] The first component attribute data is the initial attribute data related to the component sample; the attribute feature is the component feature carried in the first component attribute data. Specifically, the component library contains a variety of component samples, and different component samples have different component identifiers. For any component sample in the component library, the component definition or component document corresponding to the component sample can be located through the component identifier to extract the component attribute data from the component definition or component document. The component attribute data is parsed to extract the attribute features of the component sample, such as component style, component function, etc.

[0070] In some optional implementations, the step of acquiring first component attribute data of a component includes: locating a storage location where a component sample is located, and extracting the first component attribute data from the storage location.

[0071] The storage location is the location of the component sample in the computer device, and the component attribute data of each component sample is stored in the form of a folder. Specifically, each component sample corresponds to a corresponding storage folder, and the folder storing the component attribute data can be located according to the component identifier to extract the first component attribute data provided by the component developer for the component sample during the development process from the folder.

[0072] By locating the storage location of the component sample to extract the first component attribute data of the component sample, accurate extraction of the component attribute data is achieved, and the extraction efficiency of the component attribute data is improved.

[0073] Step S2012: Expand the attributes of the component according to the attribute characteristics to obtain second component attribute data corresponding to the component sample.

[0074] The attribute feature is used as component expansion prompt information, and the component expansion prompt information is used to expand the component attribute data to obtain second component attribute data similar to the first component attribute data. Specifically, an attribute expansion model is deployed in a computer device, and the attribute feature is input into a pre-trained attribute expansion model to expand the attributes of the component sample, so as to obtain second component attribute data with the same format as the first component attribute data but different attribute content.

[0075] The attribute expansion model is trained by taking attribute feature samples as input and component attribute data samples corresponding to the attribute feature samples as output. Here, the attribute expansion model can adopt a large language model architecture. Of course, other model architectures can also be adopted. The model structure of the attribute expansion model is not limited here. As long as the component attribute data can be expanded based on the attribute features, technicians in this field can determine it according to actual needs.

[0076] In some optional implementations, the above step S2022 includes:

[0077] Step a1, constructing extended prompt information of component samples using attribute features.

[0078] Step a2: Expand the component attributes according to the expansion prompt information to generate at least one set of second component attribute data.

[0079] When there are multiple groups of second component attribute data, the data format of each group of second component attribute data is the same, and the content of each group of second component attribute data is different.

[0080] The extended prompt information is the prompt information required to extend the component attribute data, and the attribute characteristics may be the component name, component type, component function, component purpose, etc. Specifically, the extended prompt information carries the attribute characteristics of the component, and by analyzing the attribute characteristics possessed by the component, the relevant prompt information of the component can be generated. The component attribute data is extended by extending the prompt information to generate one or more groups of second component attribute data that meet the attribute characteristics according to the extended prompt information. When there are multiple groups of second component attribute data, the data format of each group of second component attribute data is the same, but the content of each group of second component attribute data is different, thereby achieving the purpose of extending the component attribute data to make the component attribute data more diversified.

[0081] For example, the format specification and content of the extended prompt information are parsed, and 10 sets of attribute data that meet the same format specification but have different contents are generated according to the extended prompt information, so as to better understand and use the component. The specific examples are as follows:

[0082] Please generate 10 sets of component attribute data with the same format according to the following attributes. Each set of component attribute data should include attribute A and attribute B, and the content should be diverse.

[0083] ### Input properties example:

[0084] Attribute A = "AAA";

[0085] Attribute B = "BBB";

[0086] ###The generated component attribute data example format is as follows:

[0087] 1. Attribute A = "XXX", Attribute B = "YYY";

[0088] 2. Attribute A = "ZZZ", Attribute B = "WWW";

[0089] 3. Attribute A = "CCC", Attribute B = "DDD";

[0090] …

[0091] Please generate 10 sets of component attribute data.

[0092] In the above example, "AAA" and "BBB" are attribute features; "attribute A="AAA"" and "attribute B="BBB"" are extended prompt information generated based on the attribute features; "attribute A="XXX", attribute B="YYY"", "attribute A="ZZZ", attribute B="WWW"" and "attribute A="CCC", attribute B="DDD"" are second component attribute data generated based on the extended prompt information.

[0093] By using attribute features as extended prompt information to expand the first component attribute data of the component and generate one or more sets of second component attribute data, the diversity of component attribute data is improved, thereby ensuring the richness of component attribute data. By automatically generating multiple sets of second component attribute data similar to the first component attribute data, the time and labor cost required for manual collection and sorting of data are significantly reduced, and the data collection efficiency is improved.

[0094] In some optional embodiments, such as Figure 7 The above step S203 may specifically include:

[0095] Step S2031: determining a rendering position of the rendering component based on the identification information of the rendering component.

[0096] The identification information is the unique identification of the rendering component, and the rendering position is the fixed position of the rendering component in the web page. After the component rendering is completed using the component attribute data, each rendering component has corresponding identification information, and components with different identification information are rendered at different positions in the page. The rendering position of the rendering component in the page can be determined by combining the identification information of the rendering component.

[0097] Step S2032, taking a screenshot of the rendering position to obtain a component screenshot corresponding to the rendering component.

[0098] Component screenshots are component images generated for rendering components. Based on the identification information of the rendering component, the rendering position of the component in the web page can be found, and the application program interface provided by the screenshot service can be called to take a screenshot of the rendering component at a fixed position in the page, and the component screenshot of the rendering component can be captured from the page, and the captured component screenshot can be stored in the corresponding database.

[0099] The image search component model training method provided in this embodiment extracts attribute features from the first component attribute data, uses the attribute features to expand the component attributes, and obtains the corresponding second component attribute data, thereby realizing the automatic expansion of the component attribute data and facilitating the automatic construction of the component attribute data set. By automatically rendering the component attribute data to render and screenshot the component at the corresponding rendering position, the accurate capture of the component screenshot is realized, which significantly improves the accuracy of the component screenshot.

[0100] In some optional embodiments, such as Figure 8 As shown, the above step S204 may specifically include:

[0101] Step S2041, obtaining the component name corresponding to the component screenshot.

[0102] The component name is the name of the component displayed in the component screenshot, such as a switch button component, a chart component, a text display component, etc. Specifically, the component name can be obtained by a technician manually marking each component screenshot; or the component recognition model can be pre-trained using image recognition technology, and the component name corresponding to the component screenshot can be output through the component recognition model; or the component screenshot can be matched with a known component template to find a template that matches the component screenshot and output the corresponding component name. Of course, the component name can also be obtained by other means, which are not specifically limited here.

[0103] Step S2042, describing the rendering component based on the component name and component information through a preset description model to generate a component description text.

[0104] The preset description model is a pre-trained model for outputting description text. Specifically, the preset description model can adopt a multimodal large language model architecture, with component image samples and component names and component information corresponding to the component images as input samples, and description text corresponding to the component images as output samples, and the multimodal large language model architecture is trained based on the input samples and output samples to obtain a corresponding description model.

[0105] A preset description model is deployed in the computer device, and the component name and component information corresponding to each component screenshot are stored in a list. Using a few sample prompts, the component name and component information are constructed into description prompt information for generating a description text.

[0106] The following are some examples:

[0107] COMPONENT_DESC_PROMPTS="""

[0108] Receives a component screenshot and the component name of the component screenshot, and describes the component based on the component screenshot.

[0109] Description format (recommended not to exceed 20 characters).

[0110] Component type, component function, and component appearance.

[0111] Some examples are:

[0112] 1. Toggle button, collapse-expand, with arrow blue.

[0113] 2. Charts, comparison indicators, and two comparison results.

[0114] 3. Chart, funnel chart, with conversion rate.

[0115] 4. Text display, list of marketing-related activity plans.

[0116] ""

[0117] According to the description prompt information corresponding to each component screenshot in the list, request the open application interface of the preset description model in turn, describe the components in the component screenshot through the preset description model, generate component description text corresponding to each component screenshot, and store each component screenshot and its corresponding component description text in a mapping table.

[0118] In some optional implementations, the above method further includes:

[0119] Step b1, display the component description text.

[0120] Step b2: in response to the correction operation on the component description text, correct the component description text.

[0121] The technician develops a front-end page for displaying the component description text in advance, and uses the front-end page to visually display each component screenshot and its corresponding component description text. The correction operation is a modification operation performed by the technician on the component description text. If it is determined that the component description sample corresponding to the component screenshot has an error, the technician can correct the component description sample through the front-end page. Accordingly, the component description sample can respond to the technician's correction operation, correct the component description text, and obtain the correct component description sample.

[0122] By supporting the display of component description text, the component description text can be corrected according to the component screenshot to ensure the accuracy of the component description text, thereby improving the component search accuracy of subsequent image search group services.

[0123] In some optional embodiments, such as Figure 8 As shown, the above step S205 may specifically include:

[0124] Step S2051, converting the component description text into a text vector.

[0125] The text vector is a vector representation of the component description text. The text vector is used as the unique identifier of each type of component to keep the associated words in components with similar styles and functions consistent. The text vector is used to capture the semantics and features of various component description texts, so that different types of component description texts have different text vector representations.

[0126] Specifically, the component description text can be split into words or terms, and each word can be converted into a vector representation using a word vector model, and then the vector representations of all words in the component description text can be combined to form a text vector; the vectors of all words can also be added or averaged, and the vector obtained by adding or averaging the vectors can be used as the text vector; the text vector model can also be trained using a convolutional neural network architecture or a recurrent neural network architecture, and the component description text can be converted into a text vector using the trained text vector model. Of course, other methods can also be used to generate text vectors, and the method for generating text vectors is not limited here.

[0127] After generating the text vector, you can also use preprocessing techniques (such as removing stop words, stemming or word form restoration) to improve the quality and effect of the text vector, and store the final text vector in a vector database so that the text vector can be retrieved later based on the image vector.

[0128] Step S2052, constructing a mapping relationship between the image vector and the text vector of the component screenshot.

[0129] The image vector is a vector representation of the component screenshot. Specifically, the component features in the component screenshot can be identified by a pre-trained image recognition model to learn the features related to the component. Then, a feature extraction model (the feature extraction model can be trained based on a convolutional neural network architecture) is used to extract features of the components in the component screenshot to obtain an image vector corresponding to the component screenshot.

[0130] The mapping relationship represents the association relationship between the image vector and the text vector. The image vector and the text vector are mapped to the same feature space, and the image vector and the text vector are associated to generate the mapping relationship between the image vector and the text vector. For example, the mapping relationship can be constructed by using a shared hidden layer or feature map, or the image vector and the text vector can be converted to the same dimension using a linear or nonlinear transformation to obtain the mapping relationship between the image vector and the text vector.

[0131] Specifically, when constructing the mapping relationship between image vectors and text vectors, it is necessary to select an appropriate model architecture. Here, we can consider using a multimodal learning method to combine the feature information in image vectors and text vectors to further improve the accuracy and generalization ability of the mapping relationship.

[0132] Step S2053, training the graph search component model based on the mapping relationship.

[0133] The image search component model is trained using the constructed mapping relationship, the image vector corresponding to the component screenshot, and the text vector. Fig. 9The image search component model architecture shown in the figure inputs the component screenshot into the image search component model to obtain the predicted label b corresponding to the component screenshot; the component description text corresponding to the component screenshot is represented by a vector to obtain the text vector corresponding to the component screenshot, and the true label a is extracted from the text vector. The predicted label b and the true label a are compared to determine the error information between the two, and the image search component model is adjusted using the error information to make the predicted label b and the true label a as consistent as possible. Among them, the error information can be represented by an error function, which can be expressed as:

[0134]

[0135] like Fig. 9 As shown, based on the mapping relationship between the image vector and the text vector, the image vector and the text vector are mapped to the same vector. Specifically, the component screenshot is input into the image search component model for model training, so as to determine the image vector corresponding to the component screenshot through the image search component model, and based on the mapping relationship between the image vector and the text vector, the text vector is searched in the vector database through the image vector to obtain the text vector associated with the image vector.

[0136] In some optional implementations, the above step S2053 includes:

[0137] Step c1, in response to a training configuration operation for the image search component model, determining a training parameter group for the image search component model.

[0138] Step c2, in response to the script writing operation for the graph search component model, determining the training script of the graph search component model.

[0139] Step c3, using the training script to execute each set of parameters in the training parameter group to train the graph search component model based on the mapping relationship.

[0140] The training configuration operation is an operation to set training parameters based on the model training process. The training parameter group is a hyperparameter group used to realize automatic training of the graph search component model. Specifically, the training parameter group includes the number of training rounds, learning rate, batch size, number of layers, number of nodes, etc. The technicians can configure the training parameters of the graph search component model according to actual needs. Correspondingly, the computer equipment deploying the graph search component model can respond to the technician's training configuration operation on the graph search component model and obtain the corresponding training parameter group, so as to find the optimal model configuration of the graph search component model by testing different hyperparameters.

[0141] The script writing operation is an operation of writing scripts based on the model training process. The training script is a script used to realize the automatic training of the graph search component model. The technician can write the training script according to the training requirements of the graph search component model. Accordingly, the computer device deploying the graph search component model can respond to the technician's script writing operation on the graph search component model and obtain the corresponding training script.

[0142] The training script is used to automatically iterate and execute each set of parameters in the training parameter group according to the mapping relationship, so as to realize automatic execution for each set of training parameters, thereby realizing automated training of the graph search component model.

[0143] By configuring the training parameter group, writing automated training scripts, and using the training scripts to perform automated training of the graph search component model under pre-configured training parameters, the model training's reliance on manual operations is significantly reduced, and the risk of human error is reduced, thereby ensuring the standardization and consistency of the training process.

[0144] In some optional implementations, the above method further includes:

[0145] Step d1, obtain the evaluation indicators and test set for the graph search component model.

[0146] Step d2: Use the evaluation indicators to perform model testing on the graph search component model on the test set to obtain the model testing results.

[0147] Step d3, evaluating the search accuracy of the image search component model based on the model test results.

[0148] Evaluation indicators are indicators used to evaluate the prediction effect of the graph search component model. Specifically, the evaluation indicators may include the first-ranked accuracy (top1 accuracy), the second-ranked accuracy (top2 accuracy), and the third-ranked accuracy (top3 accuracy). Of course, the evaluation indicators may also include recall rate, average accuracy mean, etc. The evaluation indicators are not specifically limited here, and those skilled in the art can set them according to actual needs.

[0149] The test set is a dataset used for indicator evaluation. Specifically, the test set is a representative dataset that is similar to the actual component search scenario. The test set can use an existing public dataset or a dataset built according to specific requirements. If the test set uses a public dataset, the data can be downloaded from the relevant dataset website or repository; if the test set is a dataset built according to requirements, the relevant test data is collected and organized.

[0150] The technicians can pre-set evaluation indicators for the image search component model. After the automatic training is completed, the pre-set evaluation indicators are called to test the effect of the image search component model. The image search component model is run on the selected test set, and the accuracy of the search results of the image search component model is evaluated according to the evaluation indicators.

[0151] Taking the top 1 accuracy, top 2 accuracy, and top 3 accuracy as examples, these three evaluation indicators measure whether the first, second, and third output results of the graph search component model contain the correct component name. These three indicators are evaluated on the preset test set and the weighted average score is calculated, with weights of 0.5, 0.3, and 0.2 respectively. Since users usually pay more attention to the first recommendation of the component search results, the weighted setting is used to emphasize the importance of the top 1 accuracy, followed by the first two and third recommendations, to ensure that accurate component search information is provided in the most critical situations.

[0152] By performing automated testing on the graph search component model, the performance of the graph search component model can be tracked in real time to ensure the selection of the best graph search component model and the effectiveness of the graph search component model in different scenarios, which is conducive to improving the update efficiency and optimization efficiency of the model and improving the management efficiency of the entire model life cycle.

[0153] In some optional implementations, the above method further includes:

[0154] Step e1: If the component data is incrementally updated, the image search component model is updated using the incrementally updated component data to obtain a new version of the image search component model.

[0155] Step e2: Use the new version of the image search component model to replace the old version of the image search component model.

[0156] Incremental updates are updates to component data. Specifically, since the component styles and component types in the component library are updated in real time, the component data in the component library is monitored in real time to determine whether the component data is updated. If the component data in the component library is updated, the current component data is compared with the original component data to determine the incrementally updated component data.

[0157] Retrain or fine-tune the graph search component model using the incrementally updated component data to obtain an updated new version of the graph search component model. Assign a unique version number to the new version of the graph search component model and record relevant update information, such as update time, update content, etc. At the same time, the new version of the graph search component model can be tested and verified using the test set and evaluation indicators to ensure that the performance and accuracy of the new version of the graph search component model meet expectations.

[0158] When it is determined that the new version of the image search component model meets expectations, the update interface of the image search component model is automatically triggered, and the new version of the image search component model is deployed to the server to replace the old version of the image search component model, so that the new version of the image search component model can be used for component search in subsequent component search applications.

[0159] When there is a component update, the update mechanism of the image search component model is automatically triggered to ensure that the image search component model can be immediately replaced and applied to the image search component service after the update, which significantly reduces the need for manual intervention and improves the component search accuracy and the adaptability of the image search component model.

[0160] The training method of the image search component model provided in this embodiment combines the component name and component information corresponding to the component screenshot, and uses a multimodal large model to generate the corresponding component description text for the component screenshot, so as to use the component description text to train the image search component model. By converting the component description text into a text vector and storing it in an internal vector database, the text vector can be retrieved later. By constructing a mapping relationship between the image vector and the text vector, the image search component model is trained using the mapping relationship so that the image search component model can establish an association relationship between the component image and its component description text, so that the corresponding text vector can be retrieved from the vector database through the image vector, and the corresponding component name can be determined in combination with the text vector. The target text vector closest to the target image vector is searched through the mapping relationship to achieve efficient retrieval of similar text vectors for component images, thereby enhancing the accuracy and flexibility of component image search.

[0161] In this embodiment, a component search device is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0162] This embodiment provides a component search device, such as Fig.10 As shown, including:

[0163] The acquisition module 301 is used to acquire the target component image to be searched.

[0164] The image processing module 302 is used to perform image processing on the target component image through the image search component model to determine the target image vector corresponding to the target component image.

[0165] The search module 303 is used to search for the target text vector that is closest to the target image vector based on the mapping relationship through the image search component model.

[0166] The result determination module 304 is used to determine the component corresponding to the target text vector as the component search result corresponding to the target component image. The image search component model is trained based on component image samples and component description texts corresponding to the component image samples.

[0167] In some optional embodiments, the above device further includes:

[0168] Model training module, used to train the graph search component model.

[0169] Specifically, the model training module includes:

[0170] The attribute acquisition unit is used to acquire component attribute data corresponding to the component sample.

[0171] The rendering unit is used to render the component according to the component attribute data to obtain the rendering component corresponding to the component attribute data.

[0172] The screenshot unit is used to take screenshots of rendering components and generate component screenshots.

[0173] The description unit is used to generate component description text based on the component screenshot and component information of the rendered component.

[0174] The training unit is used to train the image search component model based on component description text and component screenshots.

[0175] In some optional implementations, the component attribute data includes first component attribute data and second component attribute data, and the attribute acquisition unit includes:

[0176] The attribute acquisition subunit is used to acquire the first component attribute data of the component sample and the attribute characteristics corresponding to the first component attribute data.

[0177] The expansion subunit is used to expand the attributes of the component sample according to the attribute characteristics to obtain the second component attribute data corresponding to the component.

[0178] In some optional implementations, the expansion unit is specifically used to construct expansion prompt information of the component sample using the attribute characteristics; expand the component attributes according to the expansion prompt information to generate at least one set of second component attribute data.

[0179] When there are multiple groups of second component attribute data, the data format of each group of second component attribute data is the same, and the content of each group of second component attribute data is different.

[0180] In some optional implementations, the screenshot unit includes:

[0181] The position determination subunit is used to determine the rendering position of the rendering component based on the identification information of the rendering component.

[0182] The component screenshot subunit is used to take a screenshot of the rendering position to obtain a component screenshot corresponding to the rendering component.

[0183] In some optional implementations, the description unit includes:

[0184] The name acquisition subunit is used to obtain the component name corresponding to the component screenshot.

[0185] The description text generation subunit is used to describe the rendering component based on the component name and component information through a preset description model and generate a component description text.

[0186] In some optional embodiments, the above device further includes:

[0187] Display module, used to display component description text.

[0188] The correction module is used to correct the component description text in response to the correction operation on the component description text.

[0189] In some optional implementations, the search unit includes:

[0190] The vector conversion subunit is used to convert the component description text into a text vector.

[0191] The mapping subunit is used to construct a mapping relationship between the image vector and the text vector of the component screenshot.

[0192] The training subunit is used to train the graph search component model based on the mapping relationship.

[0193] In some optional embodiments, the above-mentioned training sub-unit is specifically used to determine the training parameter group of the image search component model in response to the training configuration operation for the image search component model; determine the training script of the image search component model in response to the script writing operation for the image search component model; and use the training script to execute each group of parameters in the training parameter group to train the image search component model based on the mapping relationship.

[0194] In some optional embodiments, the above device further includes:

[0195] The evaluation information acquisition module is used to obtain evaluation indicators and test sets for the graph search component model.

[0196] The testing module is used to perform model testing on the graph search component model using evaluation indicators on the test set to obtain model testing results.

[0197] The model evaluation module is used to evaluate the search accuracy of the graph search component model based on the model test results.

[0198] In some optional embodiments, the above device further includes:

[0199] The update module is used to update the graph search component model using the incrementally updated component data if the component data is incrementally updated to obtain a new version of the graph search component model.

[0200] The replacement module is used to replace the old version of the image search component model with the new version of the image search component model.

[0201] The component search device in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0202] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0203] The component search device provided by the embodiment of the present disclosure uses component image samples and component description texts corresponding to the component image samples to train an image search component model, and deploys the image search component model to the server. Thus, when receiving a target component image input by a user, the component information in the target component image is extracted, and the extracted component information is input into the image search component model, so that the image search component model can perform component search according to the target component image and obtain corresponding component search results. Thereby, the effect of searching for the target component from the component library through the component image is achieved, which greatly improves the convenience in the component search and reuse process. Developers do not need to manually browse the huge business component library to find similar components, which reduces the difficulty of component search, thereby shortening the development cycle of data business scenarios and improving the development efficiency of data business scenarios.

[0204] The present disclosure also provides a computer device having the above Fig.10 The component search device shown.

[0205] See also Fig.11 , Fig.11 is a schematic diagram of a computer device provided by an optional embodiment of the present disclosure, such as Fig.11As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple groups of buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Fig.11 A processor 10 is taken as an example.

[0206] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0207] The memory 20 stores instructions executable by at least one processor 10, so that the at least one processor 10 executes the method shown in the above embodiment.

[0208] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created by the use of a computer device based on the presentation of a small program landing page, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0209] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0210] The computer device also includes an input device 30 and an output device 40. The processor 10, the memory 20, the input device 30 and the output device 40 may be connected via a bus or other means. Fig.11 The example of connecting through bus is taken in the following.

[0211] The input device 30 can receive input digital or character information, and generate key signal input related to the user settings and function control of the computer device, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, an indicator bar, one or more mouse buttons, a trackball, a joystick, etc. The output device 40 may include a display device, an auxiliary lighting device (e.g., an LED) and a tactile feedback device (e.g., a vibration motor), etc. The above-mentioned display device includes but is not limited to a liquid crystal display, a light emitting diode, a display and a plasma display. In some optional embodiments, the display device can be a touch screen.

[0212] The computer device also includes a communication interface, which is used for the computer device to communicate with other devices or a communication network.

[0213] The embodiments of the present disclosure also provide a computer-readable storage medium. The above-mentioned method according to the embodiments of the present disclosure can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium and downloaded through a network, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state drive, etc.; further, the storage medium can also include a combination of the above-mentioned types of memory. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0214] A part of the present disclosure may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present disclosure through the operation of the computer. Those skilled in the art should understand that the existence of computer program instructions in computer-readable media includes, but is not limited to, source files, executable files, installation package files, etc., and accordingly, the way in which computer program instructions are executed by a computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to the computer.

[0215] Although the embodiments of the present disclosure have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present disclosure, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A component search method, characterized in that: The method comprises: Get the target component image to be searched; Performing image processing on the target component image through the image search component model to determine the target image vector corresponding to the target component image; Searching, by the image search component model, for a target text vector that is closest to the target image vector based on the mapping relationship; Determine the component corresponding to the target text vector as the component search result corresponding to the target component image; The image search component model is trained based on component image samples and component description texts corresponding to the component image samples.

2. The method according to claim 1, characterized in that Training the image search component model includes: Get component attribute data corresponding to the component sample; Render the component according to the component attribute data to obtain a rendering component corresponding to the component attribute data; Taking a screenshot of the rendering component to generate a component screenshot; Generate component description text based on the component screenshot and component information of the rendered component; The image search component model is trained based on the component description text and the component screenshot.

3. The method according to claim 1, characterized in that The obtaining of component attribute data corresponding to the component sample includes: Acquire first component attribute data of the component sample and attribute features corresponding to the first component attribute data; Performing attribute expansion on the component sample according to the attribute feature to obtain second component attribute data corresponding to the component sample; The component attribute data includes first component attribute data and the second component attribute data.

4. The method according to claim 3, characterized in that The obtaining of first component attribute data of the component sample includes: The storage location of the component sample is located, and the first component attribute data is extracted from the storage location.

5. The method according to claim 3 or 4, characterized in that: The step of performing attribute expansion on the component sample according to the attribute feature to obtain second component attribute data corresponding to the component sample includes: Using the attribute features to construct extended prompt information of the component sample; Expand the component attributes according to the expansion prompt information to generate at least one set of the second component attribute data; When there are multiple groups of the second component attribute data, the data format of each group of the second component attribute data is the same, and the content of each group of the second component attribute data is different.

6. The method according to claim 2, characterized in that The taking a screenshot of the rendering component to generate a component screenshot includes: Determining a rendering position of the rendering component based on the identification information of the rendering component; A screenshot is taken of the rendering position to obtain a component screenshot corresponding to the rendering component.

7. The method according to claim 2, characterized in that The generating component description text based on the component screenshot and the component information of the rendered component includes: Get the component name corresponding to the component screenshot; The rendering component is described based on the component name and the component information through a preset description model to generate a component description text.

8. The method according to claim 7, characterized in that Also includes: Display the component description text; In response to a correction operation on the component description text, the component description text is corrected.

9. The method according to claim 2, characterized in that: The training of the image search component model based on the component description text and the component screenshot includes: Convert the component description text into a text vector, and construct a mapping relationship between the image vector of the component screenshot and the text vector; The image search component model is trained based on the mapping relationship.

10. The method according to claim 9, characterized in that The training of the graph search component model based on the mapping relationship includes: In response to a training configuration operation for the image search component model, determining a training parameter group for the image search component model; In response to a script writing operation for the image search component model, determining a training script for the image search component model; The training script is used to execute each group of parameters in the training parameter group to train the image search component model based on the mapping relationship.

11. The method according to claim 9 or 10, characterized in that: Also includes: Obtaining evaluation indicators and test sets for the image search component model; Performing a model test on the graph search component model using the evaluation index on the test set to obtain a model test result; The search accuracy of the image search component model is evaluated based on the model test results.

12. The method according to claim 2, characterized in that: Also includes: If the component data is incrementally updated, the image search component model is updated using the incrementally updated component data to obtain a new version of the image search component model; The new version of the image search component model is used to replace the old version of the image search component model.

13. A component search device, characterized in that: The device comprises: An acquisition module is used to obtain the target component image to be searched; An image processing module, used to perform image processing on the target component image through an image search component model to determine a target image vector corresponding to the target component image; A search module, configured to search for a target text vector that is closest to the target image vector based on the mapping relationship through the image search component model; The image search component model is trained based on component image samples and component description texts corresponding to the component image samples.

14. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the component search method according to any one of claims 1 to 12 by executing the computer instructions.

15. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the component search method according to any one of claims 1 to 12.

16. A computer program product, characterized in that The method comprises computer instructions for causing a computer to execute the component search method according to any one of claims 1 to 12.

Images