Method and system for retrieving design content based on large language model
Through the search method based on the large language model, the matching degree between the design content template and the content to be processed is calculated and the template list is filtered, which solves the problem of designers looking for inspiration in design drawing for a long time, and improves the design efficiency of design drawings and Search for recall effect.
Patent Information
- Application Number
- CN202411091692.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-09
- Publication Date
- 2024-11-08
AI Technical Summary
In design scenarios, designers spend a lot of time looking for design inspiration when drawing design drawings, resulting in inefficient design.
Using a search method based on a large language model, by obtaining the content of the pending description, calculating the matching degree between it and the design content template, filtering out the initial template list, and using the large language model to obtain the final template list from the initial template list, improving Design efficiency of design drawings.
It greatly saves designers' time and energy, improves the design efficiency of design drawings, can quickly complete the retrieval and recall of design content templates, and even accurately find templates that meet the needs when users only enter words or phrases.
Smart Images

Figure CN118916023A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of design software, and in particular to a method and system for retrieving design content based on a large language model. Background Art
[0002] In the design scenario, it is a very complicated task for designers to draw design drawings. Designers need to spend a lot of time looking for design inspiration before drawing the design drawings, which will lead to very low design efficiency.
[0003] Therefore, there is an urgent need for a method that can improve the design efficiency of design drawings. Summary of the invention
[0004] In view of this, an embodiment of the present invention provides a method and system for retrieving design content based on a large language model to improve the design efficiency of design drawings.
[0005] To achieve the above objectives, the embodiments of the present invention provide the following technical solutions:
[0006] A first aspect of an embodiment of the present invention discloses a method for retrieving design content based on a large language model, the method comprising:
[0007] Obtaining description content to be processed, wherein the description content to be processed includes: design drawing description information corresponding to the design drawing to be processed, and / or text description information input by the user;
[0008] Calculating the matching degree between the description content to be processed and each design content template, wherein the design content template carries corresponding natural language description information and key content description information;
[0009] The design content templates are screened according to the calculated matching degree to obtain an initial template list;
[0010] The description content to be processed and the initial template list are input into a preset large language model to obtain a final template list.
[0011] Preferably, the process of obtaining the design drawing description information corresponding to the design drawing to be processed includes:
[0012] Get the information of the area selected by the user on the original design;
[0013] According to the information of the area selected by the user on the original design drawing, the original design drawing is processed to obtain a design drawing to be processed, wherein the design drawing to be processed carries the information of the area selected by the user;
[0014] The design drawing to be processed is input into a preset large language model to obtain design drawing description information corresponding to the design drawing to be processed.
[0015] Preferably, calculating the matching degree between the description content to be processed and each design content template includes:
[0016] If the description content to be processed includes the design drawing description information, embedding matching is performed on the design drawing description information and the natural language description information carried by each design content template to obtain the matching degree between the design drawing description information and each of the design content templates;
[0017] If the description content to be processed includes the text description information, embedding matching is performed on the text description information and the key content description information carried by each of the design content templates to obtain the matching degree between the text description information and each of the design content templates;
[0018] If the description content to be processed includes the design drawing description information and the text description information, embedding matching is performed on the design drawing description information and the natural language description information carried by each of the design content templates to obtain the matching degree between the design drawing description information and each of the design content templates; and embedding matching is performed on the text description information and the key content description information carried by each of the design content templates to obtain the matching degree between the text description information and each of the design content templates.
[0019] Preferably, the design content templates are screened according to the calculated matching degree to obtain an initial template list, including:
[0020] If the description content to be processed includes the design drawing description information, a first list as an initial template list is screened out according to the matching degree between the design drawing description information and each of the design content templates, the first list including: the design content templates whose matching degree with the design drawing description information is greater than a first threshold;
[0021] If the description content to be processed includes the text description information, a second list as an initial template list is screened out according to the matching degree between the text description information and each of the design content templates, the second list including: the design content templates whose matching degree with the text description information is greater than a second threshold;
[0022] If the description content to be processed includes the design drawing description information and the text description information, the first list is filtered out according to the matching degree between the design drawing description information and each of the design content templates, and the second list is filtered out according to the matching degree between the text description information and each of the design content templates; the first list and the second list are merged and deduplicated to obtain an initial template list.
[0023] Preferably, the description content to be processed and the initial template list are input into a preset large language model to obtain a final template list, including:
[0024] The description content to be processed and the initial template list are input into a preset large language model, and a question is asked to the large language model in combination with a preset first prompt, so that the large language model filters out a final template list from the initial template list.
[0025] Preferably, the design drawing to be processed is input into a preset large language model to obtain the design drawing description information corresponding to the design drawing to be processed, including:
[0026] The design drawing to be processed is input into a preset large language model, and a question is asked to the large language model in combination with a preset second prompt, so that the large language model outputs design drawing description information corresponding to the design drawing to be processed.
[0027] A second aspect of an embodiment of the present invention discloses a system for retrieving design content based on a large language model, the system comprising:
[0028] An acquisition unit, used for acquiring description content to be processed, wherein the description content to be processed includes: design drawing description information corresponding to the design drawing to be processed, and / or text description information input by a user;
[0029] A calculation unit, used to calculate the matching degree between the description content to be processed and each design content template, wherein the design content template carries corresponding natural language description information and key content description information;
[0030] A screening unit, configured to screen the design content templates according to the calculated matching degree to obtain an initial template list;
[0031] The input unit is used to input the description content to be processed and the initial template list into a preset large language model to obtain a final template list.
[0032] Preferably, the acquisition unit includes:
[0033] The acquisition module is used to obtain information about the area selected by the user on the original design drawing;
[0034] A processing module, used to process the original design drawing according to the information of the area selected by the user on the original design drawing, so as to obtain a design drawing to be processed, wherein the design drawing to be processed carries the information of the area selected by the user;
[0035] The input module is used to input the design drawing to be processed into a preset large language model to obtain the design drawing description information corresponding to the design drawing to be processed.
[0036] Preferably, the calculation unit comprises:
[0037] A first matching module is used for, if the description content to be processed includes the design drawing description information, performing embedding matching between the design drawing description information and the natural language description information carried by each design content template, to obtain the matching degree between the design drawing description information and each of the design content templates;
[0038] A second matching module is used for, if the description content to be processed includes the text description information, performing embedding matching between the text description information and the key content description information carried by each of the design content templates to obtain a matching degree between the text description information and each of the design content templates;
[0039] The third matching module is used to perform embedding matching on the design drawing description information and the natural language description information carried by each of the design content templates to obtain the matching degree between the design drawing description information and each of the design content templates if the description content to be processed includes the design drawing description information and the text description information, and to perform embedding matching on the text description information and the key content description information carried by each of the design content templates to obtain the matching degree between the text description information and each of the design content templates.
[0040] Preferably, the screening unit comprises:
[0041] A first screening module is used for screening out a first list as an initial template list according to the matching degree between the design drawing description information and each of the design content templates if the description content to be processed includes the design drawing description information, wherein the first list includes: the design content templates whose matching degree with the design drawing description information is greater than a first threshold value;
[0042] A second screening module is used for screening out a second list as an initial template list according to the matching degree between the text description information and each of the design content templates if the description content to be processed includes the text description information, wherein the second list includes: the design content templates whose matching degree with the text description information is greater than a second threshold value;
[0043] The third filtering module is used to filter out the first list according to the matching degree between the design drawing description information and each of the design content templates, and to filter out the second list according to the matching degree between the text description information and each of the design content templates if the description content to be processed includes the design drawing description information and the text description information; and to merge and deduplicate the first list and the second list to obtain an initial template list.
[0044] Based on the above-mentioned embodiment of the present invention, a method and system for retrieving design content based on a large language model is provided. The method is as follows: obtaining the description content to be processed; calculating the matching degree between the description content to be processed and each design content template; screening the design content templates according to the calculated matching degree to obtain an initial template list; inputting the description content to be processed and the initial template list into a preset large language model to obtain a final template list. In this scheme, the matching degree between the description content to be processed and each design content template is calculated first, and the design content templates are screened according to the calculated matching degree to obtain an initial template list; then the large language model is used to obtain the final template list from the initial template list, and the design content templates that match the description content to be processed are retrieved through the large language model, thereby improving the design efficiency of the design drawing. BRIEF DESCRIPTION OF THE DRAWINGS
[0045] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.
[0046] Figure 1 A flowchart of a method for retrieving design content based on a large language model provided by an embodiment of the present invention;
[0047] Figure 2 An example diagram of a design interface provided by an embodiment of the present invention;
[0048] Figure 3 An example diagram of a design diagram to be processed provided by an embodiment of the present invention;
[0049] Figure 4An example diagram of a second prompt provided in an embodiment of the present invention;
[0050] Figure 5 An example diagram of design drawing description information provided by an embodiment of the present invention;
[0051] Figure 6 An example diagram of the system prompt in the first prompt provided in an embodiment of the present invention;
[0052] Figure 7 An example diagram of a userprompt in a first prompt provided in an embodiment of the present invention;
[0053] Figure 8 Another example diagram of the userprompt in the first prompt provided in an embodiment of the present invention;
[0054] Fig. 9 Another flow chart of a method for retrieving design content based on a large language model provided by an embodiment of the present invention;
[0055] Fig.10 A structural block diagram of a system for retrieving design content based on a large language model provided in an embodiment of the present invention. DETAILED DESCRIPTION
[0056] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0057] In this application, the terms "comprises", "comprising" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprising a ..." does not exclude the presence of other identical elements in the process, method, article or device comprising the element.
[0058] In the design scenario, it is a very complicated task for designers to draw design drawings. On the one hand, designers need to complete the drawing of design content layer by layer, and often a UI page takes several hours or even days to complete the design. On the other hand, the quality of the design drawings is very dependent on the experience and ability of the designer. Designers need to spend a lot of time looking for design inspiration before drawing the design drawings. The above two reasons will lead to very low design efficiency of the design drawings.
[0059] With the development of artificial intelligence, artificial intelligence can help people complete their work more efficiently in some specific fields. The inventor found in his research that using artificial intelligence technology to assist designers in retrieving design content can greatly save designers' time and energy, thereby effectively improving the design efficiency of design drawings.
[0060] Therefore, an embodiment of the present invention provides a method and system for retrieving design content based on a large language model, which first calculates the matching degree between the description content to be processed and each design content template, and then screens the design content templates according to the calculated matching degree to obtain an initial template list; then uses the large language model to obtain the final template list from the initial template list, and uses the large language model to retrieve the design content template that matches the description content to be processed, thereby improving the design efficiency of the design drawing.
[0061] See also Figure 1 , shows a flow chart of a method for retrieving design content based on a large language model provided by an embodiment of the present invention, the method comprising:
[0062] Step S101: Obtain description content to be processed.
[0063] In the specific implementation of step S101, the description content to be processed is obtained, and the description content to be processed includes: design drawing description information corresponding to the design drawing to be processed (recorded as design_description), and / or text description information input by the user (recorded as user_input).
[0064] It should be noted that the information contained in the description content to be processed is related to the user's operation on the design interface;
[0065] If the user only opens the original design drawing on the design interface and selects a certain area on the original design drawing (the user does not enter text description information), the description content to be processed obtained only includes the design drawing description information corresponding to the design drawing to be processed;
[0066] If the user only inputs text description information in the design interface (the user does not open the original design drawing or does not select a certain area on the original design drawing), the description content to be processed obtained only includes the text description information input by the user;
[0067] If the user opens the original design drawing on the design interface and selects an area on the original design drawing, and the user also enters text description information in the design interface, the description content to be processed obtained includes: the design drawing description information corresponding to the design drawing to be processed and the text description information entered by the user.
[0068] The original design drawing mentioned above may be a design drawing being drawn by a user.
[0069] In some embodiments, the specific method of obtaining the design drawing description information corresponding to the design drawing to be processed is: obtaining the information of the area selected by the user on the original design drawing; processing the original design drawing according to the information of the area selected by the user on the original design drawing to obtain the design drawing to be processed, which carries the information of the area selected by the user; inputting the design drawing to be processed into a preset large language model to obtain the design drawing description information corresponding to the design drawing to be processed.
[0070] In some other embodiments, the specific implementation method of obtaining the design drawing description information corresponding to the design drawing to be processed is: inputting the design drawing to be processed into a preset large language model (LLM), asking questions to the large language model in combination with a preset second prompt, so that the large language model outputs the design drawing description information corresponding to the design drawing to be processed.
[0071] That is to say, the design drawing to be processed is input into the large language model, and questions are asked to the large language model through the second prompt, so that the large language model can recognize the design drawing to be processed and summarize the design drawing description information (design_description) corresponding to the design drawing to be processed, and the design drawing description information is expressed in natural language.
[0072] It should be noted that the information of the area selected by the user on the original design drawing includes the area position and area size, etc. The information of the area selected by the user can reflect what type of design content the user may expect to obtain.
[0073] For example, assuming that the user selects a wider and shorter area at the top of the original design drawing, then information such as the area position and area size of the area reflects that the user may expect to obtain a top navigation bar, and the top navigation bar is the design content.
[0074] For another example, suppose the user selects a wider and shorter area at the bottom of the original design, then information such as the area position and area size of the area reflects that the user may expect to obtain a bottom tab bar.
[0075] For another example: suppose the user selects a larger area in the middle of the original design drawing, then information such as the area position and area size of the area reflects that the user may expect to obtain a list of contents.
[0076] Therefore, the information of the area selected by the user in the design drawing to be processed can help the large language model analyze what design content the user wants.
[0077] It should be further explained that the specific implementation method of "processing the original design drawing to obtain the design drawing to be processed" is: since the "area selected by the user on the original design drawing" belongs to the data of the upper layer of the original design drawing (not the original design drawing itself), it is necessary to mark the area selected by the user on the original design drawing according to the information of the area selected by the user on the original design drawing (equivalent to merging the original design drawing and the area selected by the user), so as to obtain the design drawing to be processed.
[0078] To better understand how to obtain the description content to be processed, take the description content to be processed including the design drawing description information and text description information as an example, combined with Figure 2 and Figure 3 Give an example.
[0079] like Figure 2 As can be seen from the design interface shown, the user selects an area on the original design image displayed on the design interface ( Figure 2 The dotted area in the design interface), at this time, the information of the area selected by the user is obtained; the user enters the text description information "a travel list" in the operation panel on the right side of the original design drawing in the design interface.
[0080] After recognizing that the user has selected an area on the original design, the original design is processed again according to the information of the area selected by the user (area position and area size), and the area selected by the user is marked on the original design, so as to obtain the following Figure 3 The design drawing to be processed is shown. Figure 3 The thick solid line area in the design drawing to be processed is the area marked on the original design drawing and selected by the user.
[0081] Will Figure 3 The design drawing to be processed shown is input into the large language model, and a question is asked to the large language model through a prompt (the second prompt), so that the large language model recognizes the design drawing to be processed and summarizes the design drawing description information corresponding to the design drawing to be processed expressed in natural language.
[0082] For example, the second prompt and the design diagram to be processed are input into the large language model, and the large language model outputs the following content:
[0083] “{"design_analysis":"This is an xxx page (first-level / second-level page) of an xxx application, which contains xxx elements. The red box is located at the xxx position of the page. This area is usually used for xxx","design_description":"Based on the analysis of the page, the red box should be filled with an xxx content and have xxx functions"}".
[0084] The “design_description” in the above example is the design drawing description information corresponding to the design drawing to be processed output by the large language model.
[0085] It is worth noting that in actual operation, the two situations of "the user selects an area on the original design drawing" and "the user inputs text description information" do not necessarily exist at the same time. The specific information in the description content to be processed can be obtained according to the actual situation;
[0086] That is to say, if the two situations of "the user selects an area on the original design drawing" and "the user inputs text description information" exist at the same time, then the description content to be processed includes the design drawing description information and the text description information;
[0087] If only one of the two situations, "the user selects an area on the original design drawing" and "the user inputs text description information", exists, then the description content to be processed includes the design drawing description information or the text description information.
[0088] Step S102: Calculate the matching degree between the description content to be processed and each design content template.
[0089] It should be noted that the design content template (template) carries the corresponding natural language description information (recorded as template_description) and key content description information (recorded as template_keywords);
[0090] For each design content template, the key content description carried by the design content template is: a keyword (one or more keywords) or a phrase summarized from the natural language description information carried by the design content template.
[0091] Specifically, two key information, natural language description information (template_description) and key content description information (template_keywords), are added to each pre-configured design content template in advance; the natural language description information carried by the design content template describes the design content template in easy-to-understand natural language, and the natural language description information is usually a natural language content; the key content description information carried by the design content template is summarized from the natural language description information carried by the design content template;
[0092] The design content template is fully described through the method of "natural language description information + key content description information".
[0093] In the specific implementation of step S102, after obtaining the description content to be processed, the matching degree between the description content to be processed and each design content template is calculated respectively;
[0094] It should be noted that, as can be seen from the content of step S101, the description content to be processed includes: the design drawing description information corresponding to the design drawing to be processed, and / or the text description information input by the user;
[0095] Therefore, when calculating the matching degree, it is mainly divided into the following three matching degree calculation situations.
[0096] The first matching degree calculation situation: if the description content to be processed includes design drawing description information, embedding matching is performed between the design drawing description information and the natural language description information carried by each design content template to obtain the matching degree between the design drawing description information and each design content template.
[0097] That is to say, if the description content to be processed only includes design drawing description information (equivalent to only the existence of design_description), the design drawing description information is respectively embedded and matched with the natural language description information (template_description) carried by each design content template, so as to obtain the matching degree between the design drawing description information and each design content template. The matching degree is also called the recall score.
[0098] It should be noted that the large language model (LLM) has excellent performance in generating paragraph descriptions. By using the multimodal capabilities of the large language model to generate design_description, and then embedding matching between design_description and template_description, which is also a paragraph description, and using the calculated matching degree to retrieve the design content template, the retrieval recall effect of the design content template can be effectively improved.
[0099] The second matching degree calculation situation: if the description content to be processed includes text description information, embedding matching is performed between the text description information and the key content description information carried by each design content template to obtain the matching degree between the text description information and each design content template.
[0100] That is to say, if the description content to be processed only includes text description information (equivalent to only user_input), the text description information is embedded and matched with the key content description information (template_keywords) carried by each design content template to obtain the matching degree between the text description information and each design content template.
[0101] It should be noted that users usually only enter simple words or phrases when entering text description information, so embedding matching is performed between user_input and template_keywords, which are also words (or phrases), and the calculated matching degree is used to retrieve the design content template, which can effectively improve the retrieval recall effect of the design content template.
[0102] The third matching degree calculation situation: If the description content to be processed includes design drawing description information and text description information, the design drawing description information is embedded and matched with the natural language description information carried by each design content template to obtain the matching degree between the design drawing description information and each design content template, and the text description information is embedded and matched with the key content description information carried by each design content template to obtain the matching degree between the text description information and each design content template.
[0103] That is to say, if the description content to be processed includes both design drawing description information and text description information, that is, if design_description and user_input exist at the same time, the design drawing description information is respectively embedded and matched with the natural language description information (template_description) carried by each design content template, so as to obtain the matching degree between the design drawing description information and each design content template; at the same time, the text description information is respectively embedded and matched with the key content description information (template_keywords) carried by each design content template, so as to obtain the matching degree between the text description information and each design content template.
[0104] Step S103: Screening the design content templates according to the calculated matching degree to obtain an initial template list.
[0105] In the specific implementation of step S103, after the matching degree is calculated, all design content templates are screened according to the calculated matching degree, so as to obtain an initial template list, which includes the screened multiple design content templates.
[0106] It can be understood that, as can be seen from the content of the above step S102, the calculated matching degree may only include: the matching degree between the design drawing description information and each design content template;
[0107] The calculated matching degree may also only include: the matching degree between the text description information and each design content template;
[0108] The calculated matching degree may also include: the matching degree between the design drawing description information and each design content template, and the matching degree between the text description information and each design content template;
[0109] Therefore, when screening design content templates according to the calculated matching degree, it is mainly divided into the following three screening situations.
[0110] The first screening situation: if the description content to be processed includes design drawing description information, a first list is screened out as an initial template list according to the matching degree between the design drawing description information and each design content template. The first list includes: design content templates whose matching degree with the design drawing description information is greater than a first threshold.
[0111] That is, if the description content to be processed only includes the design drawing description information, then the calculated matching degree only includes: the matching degree between the design drawing description information and each design content template;
[0112] At this time, according to the matching degree between the design drawing description information and each design content template, each design content template is screened, so as to screen out a first list (referred to as design_emb_list) as an initial template list (referred to as emb_list);
[0113] In other words, if the description content to be processed only includes design drawing description information, the first list obtained by screening is the initial template list (ie, emb_list=design_emb_list).
[0114] It should be noted that the matching degree between each design content template and the design drawing description information in the first list is greater than the first threshold (denoted as x1), and the corresponding upper limit of the number of the first list is pre-set (denoted as y), that is, the first list can accommodate a maximum of y design content templates.
[0115] The values of the first threshold (x1) and the upper limit of quantity (y) can be set according to actual conditions and are not limited here.
[0116] The second screening situation: If the description content to be processed includes text description information, a second list is screened out as the initial template list according to the matching degree between the text description information and each design content template. The second list includes: design content templates whose matching degree with the text description information is greater than a second threshold.
[0117] That is, if the description content to be processed only includes text description information, then the calculated matching degree only includes: the matching degree between the text description information and each design content template;
[0118] At this time, according to the matching degree between the text description information and each design content template, each design content template is screened, so as to screen out a second list (recorded as user_input_emb_list) as the initial template list (recorded as emb_list);
[0119] In other words, if the description content to be processed only includes text description information, the second list obtained by screening is the initial template list (ie, emb_list=user_input_emb_list).
[0120] It should be noted that the matching degree between each design content template and the text description information in the second list is greater than the second threshold (denoted as x2), and the corresponding upper limit of the second list is pre-set (denoted as y), that is, the second list can accommodate a maximum of y design content templates.
[0121] The values of the second threshold (x2) and the upper limit of quantity (y) can be set according to actual conditions and are not limited here.
[0122] For example: assuming x2=0.6, y=30, the second list can accommodate a maximum of 30 design content templates, and the matching degree between these 30 design content templates in the second list and the text description information is greater than 0.6; that is, the second list contains a maximum of 30 design content templates with a matching degree greater than 0.6.
[0123] It should be noted that the first threshold (x1) and the second threshold (x2) mentioned above can take the same value, or the first threshold and the second threshold can take different values.
[0124] The third screening situation: If the description content to be processed includes design drawing description information and text description information, the first list is screened out according to the matching degree between the design drawing description information and each design content template, and the second list is screened out according to the matching degree between the text description information and each design content template; the first list and the second list are merged and deduplicated to obtain an initial template list.
[0125] That is to say, if the description content to be processed includes both design drawing description information and text description information, then the calculated matching degree includes: the matching degree between the design drawing description information and each design content template, and the matching degree between the text description information and each design content template;
[0126] At this time, according to the matching degree between the design drawing description information and each design content template, each design content template is screened to obtain a first list (design_emb_list);
[0127] At the same time, according to the matching degree between the text description information and each design content template, each design content template is screened to obtain a second list (user_input_emb_list);
[0128] The filtered first list and the second list are merged and deduplicated to obtain an initial template list (emb_list).
[0129] In other words, if the description content to be processed includes both design drawing description information and text description information, then after deduplication and merging of the first list and the second list, an initial template list (ie, emb_list=design_emb_list+user_input_emb_list) can be obtained.
[0130] The above process of obtaining the initial template list is equivalent to the first round of initial screening of the design content templates. The initial template list obtained by screening contains design content templates that meet user needs, which can not only improve the retrieval recall effect, but also reduce the amount of data in the second round of screening.
[0131] Step S104: input the description content to be processed and the initial template list into a preset large language model to obtain a final template list.
[0132] In the specific implementation of step S104, after the initial template list is screened, the description content to be processed and the initial template list are input into the large language model to obtain a final template list, which includes: the design content templates obtained by further screening the initial template list.
[0133] Specifically, the description content to be processed and the initial template list are input into the large language model, and the large language model is asked questions in combination with the preset first prompt, so that the large language model selects the final template list from the initial template list;
[0134] That is to say, the description content to be processed and the initial template list are input into the large language model, and questions are asked to the large language model in combination with the first prompt, so that the large language model performs a second round of screening on the design content templates in the initial template list, thereby screening out the final template list.
[0135] It should be noted that if the description content to be processed only includes the design drawing description information (equivalent to only the existence of design_description), then the design_description and the initial template list are input into the large language model to filter out the final template list;
[0136] If the description to be processed only includes text description information (equivalent to only user_input), then user_input and the initial template list are input into the large language model to filter out the final template list;
[0137] If the description to be processed includes both design_description and user_input, then user_input, design_description and the initial template list are input into the large language model to filter out the final template list;
[0138] Among them, since user_input is the demand actively input by the user, the weight of user_input input into the big language model is higher than the weight of design_description input into the big language model.
[0139] In some embodiments, a final template list obtained by filtering the large language model is fed back to the user. For example, multiple design content templates in the final template list can be displayed on the design interface for user selection.
[0140] In an embodiment of the present invention, the matching degree between the description content to be processed and each design content template is calculated first, and the design content templates are screened according to the calculated matching degree to obtain an initial template list; then, a large language model is used to obtain a final template list from the initial template list, and the large language model is used to retrieve the design content template that matches the description content to be processed, thereby improving the design efficiency of the design drawing.
[0141] Through the above-mentioned embodiments of the present invention Figure 1 As can be seen from the content, the second prompt is used to enable the large language model to identify the design drawing to be processed and summarize the design drawing description information (design_description) corresponding to the design drawing to be processed. In practical applications, one example of the second prompt is as follows Figure 4 As shown, Figure 4 The second prompt in is composed of userprompt;
[0142] After the image to be processed is input into the large language model, Figure 4 The second prompt shown asks a question to the large language model, and the large language model outputs Figure 5 Design drawing description information (design_description) is shown.
[0143] The first prompt is used to enable the large language model to filter out the final template list from the initial template list. In practical applications, the first prompt is composed of systemprompt and userprompt. An example of systemprompt in the first prompt is as follows: Figure 6 As shown, one example of the userprompt in the first prompt is as follows Figure 7 shown.
[0144] It is worth noting that if Figure 7 As shown in the content, the userprompt in the first prompt carries text description information (user_input) or design description information (design_description). When the final template list is screened out using the large language model, the specific content of the userprompt in the first prompt is as follows: Figure 8 As shown; the first prompt is used to ask questions to the large language model, and the specific content of the final template list obtained is "JSON[1,2,5]", where 1, 2 and 5 in "JSON[1,2,5]" respectively represent the IDs of the design content templates.
[0145] To further facilitate understanding of the above embodiments of the present invention Figure 1 content, through Fig. 9 Another flowchart of a method for retrieving design content based on a large language model is shown for example;
[0146] It should be noted that Fig. 9 In the example given, the description content to be processed includes both design description information (design_description) and text description information (user_input). Fig. 9 The following steps are involved:
[0147] Step S901: Preset the design content template.
[0148] Step S902: adding natural language description information (template_description) and key content description information (template_keywords) to each design content template.
[0149] Step S903: The user selects any area.
[0150] Step S904: Obtain text description information (user_input).
[0151] Step S905: Perform embedding matching on user_input and template_keywords.
[0152] Step S906: Filter out the second list (user_input_emb_list).
[0153] Step S907: Obtain the original design drawing.
[0154] Step S908: Process the original design drawing according to the area selected by the user to obtain a design drawing to be processed.
[0155] Step S909: Obtain design description information (design_description) through the large language model (LLM).
[0156] Step S910: perform embedding matching on design_description and template_keywords.
[0157] Step S911: Filter out the first list (design_emb_list).
[0158] It should be noted that “step S904 - step S906” and “step S907 - step S911” are two step branches executed in parallel.
[0159] Step S912: merge and remove duplicates of design_emb_list and user_input_emb_list to obtain an initial template list (emb_list).
[0160] Step S913: input user_input, design_description and emb_list into LLM for secondary screening to obtain the final template list.
[0161] Step S914: Feedback the final template list to the user.
[0162] It should be noted that the execution principle of step S901 to step S914 can be found in the above embodiment of the present invention. Figure 1 The contents in will not be repeated here.
[0163] The above is the relevant description of this solution. This solution will achieve the following beneficial effects in practical application:
[0164] 1. Solve the token limitation problem when using a large language model. When there are a large number of design content templates, the retrieval and recall of design content templates can be completed quickly.
[0165] 2. Even if the user only inputs a word or phrase (equivalent to the description content to be processed only including the text description content input by the user), the design content template that meets the user's needs can be accurately found;
[0166] 3. Even if the user does not enter words and phrases, the design content template that meets the user's needs can be accurately found through the user's operations on the original design drawing (equivalent to the description content to be processed only including the design drawing description information).
[0167] Corresponding to the method for retrieving design content based on a large language model provided in the above embodiment of the present invention, see Fig.10 , the embodiment of the present invention also provides a structural block diagram of a system for retrieving design content based on a large language model, the system comprising: an acquisition unit 100, a calculation unit 200, a screening unit 300 and an input unit 400;
[0168] The acquisition unit 100 is used to acquire the description content to be processed, and the description content to be processed includes: the design drawing description information corresponding to the design drawing to be processed, and / or the text description information input by the user.
[0169] The calculation unit 200 is used to calculate the matching degree between the description content to be processed and each design content template, and the design content template carries corresponding natural language description information and key content description information.
[0170] The screening unit 300 is used to screen the design content templates according to the calculated matching degree to obtain an initial template list.
[0171] The input unit 400 is used to input the description content to be processed and the initial template list into a preset large language model to obtain a final template list.
[0172] In a specific implementation, the input unit 400 is specifically used to: input the description content to be processed and the initial template list into a preset large language model, and ask questions to the large language model in combination with a preset first prompt, so that the large language model filters out a final template list from the initial template list.
[0173] In an embodiment of the present invention, the matching degree between the description content to be processed and each design content template is calculated first, and the design content templates are screened according to the calculated matching degree to obtain an initial template list; then, a large language model is used to obtain a final template list from the initial template list, and the large language model is used to retrieve the design content template that matches the description content to be processed, thereby improving the design efficiency of the design drawing.
[0174] Preferably, combined Fig.10 The content shown, the acquisition unit 100 includes an acquisition module, a processing module and an input module, and the execution principle of each module is as follows:
[0175] The acquisition module is used to obtain information about the area selected by the user on the original design drawing.
[0176] The processing module is used to process the original design drawing according to the information of the area selected by the user on the original design drawing to obtain the design drawing to be processed, and the design drawing to be processed carries the information of the area selected by the user.
[0177] The input module is used to input the design drawing to be processed into a preset large language model to obtain the design drawing description information corresponding to the design drawing to be processed.
[0178] In the specific implementation, the input module is specifically used to: input the design drawing to be processed into a preset large language model, ask questions to the large language model in combination with the preset second prompt, so that the large language model outputs the design drawing description information corresponding to the design drawing to be processed.
[0179] Preferably, combined Fig.10 As shown in the figure, the calculation unit 200 includes a first matching module, a second matching module and a third matching module, and the execution principle of each module is as follows:
[0180] The first matching module is used to perform embedding matching between the design drawing description information and the natural language description information carried by each design content template if the description content to be processed includes design drawing description information, so as to obtain the matching degree between the design drawing description information and each design content template.
[0181] The second matching module is used to perform embedding matching between the text description information and the key content description information carried by each design content template if the description content to be processed includes text description information, so as to obtain the matching degree between the text description information and each design content template.
[0182] The third matching module is used to, if the description content to be processed includes design drawing description information and text description information, embedding match the design drawing description information with the natural language description information carried by each design content template to obtain the matching degree between the design drawing description information and each design content template, and embedding match the text description information with the key content description information carried by each design content template to obtain the matching degree between the text description information and each design content template.
[0183] Preferably, combined Fig.10 The content shown, the screening unit 300 includes a first screening module, a second screening module and a third screening module, and the execution principle of each module is as follows:
[0184] The first screening module is used to screen out a first list as an initial template list according to the matching degree between the design drawing description information and each design content template if the description content to be processed includes design drawing description information. The first list includes: design content templates whose matching degree with the design drawing description information is greater than a first threshold.
[0185] The second screening module is used to screen out a second list as an initial template list according to the matching degree between the text description information and each design content template if the description content to be processed includes text description information. The second list includes: design content templates whose matching degree with the text description information is greater than a second threshold.
[0186] The third filtering module is used to filter out the first list according to the matching degree between the design drawing description information and each design content template, and to filter out the second list according to the matching degree between the text description information and each design content template if the description content to be processed includes design drawing description information and text description information; and to merge and deduplicate the first list and the second list to obtain an initial template list.
[0187] To summarize, an embodiment of the present invention provides a method and system for retrieving design content based on a large language model. The method first calculates the matching degree between the description content to be processed and each design content template, and then screens the design content templates according to the calculated matching degree to obtain an initial template list. Then, the large language model is used to obtain the final template list from the initial template list, and the large language model is used to retrieve the design content template that matches the description content to be processed, thereby improving the design efficiency of the design drawings.
[0188] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can refer to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system or system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can refer to the partial description of the method embodiment. The system and system embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without creative work.
[0189] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.
[0190] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A method for retrieving design content based on a large language model, characterized in that: The method comprises: Obtaining description content to be processed, wherein the description content to be processed includes: design drawing description information corresponding to the design drawing to be processed, and / or text description information input by the user; Calculating the matching degree between the description content to be processed and each design content template, wherein the design content template carries corresponding natural language description information and key content description information; The design content templates are screened according to the calculated matching degree to obtain an initial template list; The description content to be processed and the initial template list are input into a preset large language model to obtain a final template list.
2. The method according to claim 1, characterized in that The process of obtaining the design drawing description information corresponding to the design drawing to be processed includes: Get the information of the area selected by the user on the original design; According to the information of the area selected by the user on the original design drawing, the original design drawing is processed to obtain a design drawing to be processed, wherein the design drawing to be processed carries the information of the area selected by the user; The design drawing to be processed is input into a preset large language model to obtain design drawing description information corresponding to the design drawing to be processed.
3. The method according to claim 1, characterized in that Calculating the matching degree between the description content to be processed and each design content template includes: If the description content to be processed includes the design drawing description information, embedding matching is performed on the design drawing description information and the natural language description information carried by each design content template to obtain the matching degree between the design drawing description information and each of the design content templates; If the description content to be processed includes the text description information, embedding matching is performed on the text description information and the key content description information carried by each of the design content templates to obtain the matching degree between the text description information and each of the design content templates; If the description content to be processed includes the design drawing description information and the text description information, embedding matching is performed on the design drawing description information and the natural language description information carried by each of the design content templates to obtain the matching degree between the design drawing description information and each of the design content templates; and embedding matching is performed on the text description information and the key content description information carried by each of the design content templates to obtain the matching degree between the text description information and each of the design content templates.
4. The method according to claim 3, characterized in that The design content templates are screened according to the calculated matching degree to obtain an initial template list, including: If the description content to be processed includes the design drawing description information, a first list as an initial template list is screened out according to the matching degree between the design drawing description information and each of the design content templates, the first list including: the design content templates whose matching degree with the design drawing description information is greater than a first threshold; If the description content to be processed includes the text description information, a second list as an initial template list is screened out according to the matching degree between the text description information and each of the design content templates, the second list including: the design content templates whose matching degree with the text description information is greater than a second threshold; If the description content to be processed includes the design drawing description information and the text description information, the first list is filtered out according to the matching degree between the design drawing description information and each of the design content templates, and the second list is filtered out according to the matching degree between the text description information and each of the design content templates; the first list and the second list are merged and deduplicated to obtain an initial template list.
5. The method according to any one of claims 1 to 4, characterized in that: The description content to be processed and the initial template list are input into a preset large language model to obtain a final template list, including: The description content to be processed and the initial template list are input into a preset large language model, and a question is asked to the large language model in combination with a preset first prompt, so that the large language model filters out a final template list from the initial template list.
6. The method according to claim 2, characterized in that Inputting the design drawing to be processed into a preset large language model to obtain design drawing description information corresponding to the design drawing to be processed, including: The design drawing to be processed is input into a preset large language model, and a question is asked to the large language model in combination with a preset second prompt, so that the large language model outputs design drawing description information corresponding to the design drawing to be processed.
7. A system for retrieving design content based on a large language model, characterized in that: The system comprises: An acquisition unit, used for acquiring description content to be processed, wherein the description content to be processed includes: design drawing description information corresponding to the design drawing to be processed, and / or text description information input by a user; A calculation unit, used to calculate the matching degree between the description content to be processed and each design content template, wherein the design content template carries corresponding natural language description information and key content description information; A screening unit, configured to screen the design content templates according to the calculated matching degree to obtain an initial template list; The input unit is used to input the description content to be processed and the initial template list into a preset large language model to obtain a final template list.
8. The system according to claim 7, characterized in that The acquisition unit comprises: The acquisition module is used to obtain information about the area selected by the user on the original design drawing; A processing module, used to process the original design drawing according to the information of the area selected by the user on the original design drawing, so as to obtain a design drawing to be processed, wherein the design drawing to be processed carries the information of the area selected by the user; The input module is used to input the design drawing to be processed into a preset large language model to obtain the design drawing description information corresponding to the design drawing to be processed.
9. The system according to claim 7, characterized in that The computing unit comprises: A first matching module is used for, if the description content to be processed includes the design drawing description information, performing embedding matching between the design drawing description information and the natural language description information carried by each design content template, to obtain the matching degree between the design drawing description information and each of the design content templates; A second matching module is used for, if the description content to be processed includes the text description information, performing embedding matching between the text description information and the key content description information carried by each of the design content templates to obtain a matching degree between the text description information and each of the design content templates; The third matching module is used to perform embedding matching on the design drawing description information and the natural language description information carried by each of the design content templates to obtain the matching degree between the design drawing description information and each of the design content templates if the description content to be processed includes the design drawing description information and the text description information, and to perform embedding matching on the text description information and the key content description information carried by each of the design content templates to obtain the matching degree between the text description information and each of the design content templates.
10. The system according to claim 9, characterized in that The screening unit comprises: A first screening module is used for screening out a first list as an initial template list according to the matching degree between the design drawing description information and each of the design content templates if the description content to be processed includes the design drawing description information, wherein the first list includes: the design content templates whose matching degree with the design drawing description information is greater than a first threshold value; A second screening module is used for screening out a second list as an initial template list according to the matching degree between the text description information and each of the design content templates if the description content to be processed includes the text description information, wherein the second list includes: the design content templates whose matching degree with the text description information is greater than a second threshold value; The third filtering module is used to filter out the first list according to the matching degree between the design drawing description information and each of the design content templates, and to filter out the second list according to the matching degree between the text description information and each of the design content templates if the description content to be processed includes the design drawing description information and the text description information; and to merge and deduplicate the first list and the second list to obtain an initial template list.
