Human-machine mutual cognition concept design method and device based on deep learning and knowledge management

Abstract

The invention discloses a man-machine mutual cognition concept design method and device based on deep learning and knowledge management, and the method comprises the steps: collecting operation data in a concept design process of a designer, and obtaining a design image and a design description; marking a design image and a design description, and mapping the design image and the design description to a design intention to obtain a private design multi-modal data set of a designer; dividing the private design multi-modal data set into a training data set and a test data set; training the deep learning network by using the training data set to obtain a model pre-training weight of the recognition model; calling a model pre-training weight, and performing design intention recognition on the test set through the recognition model to obtain a to-be-fed-back design intention; using a design knowledge matching framework to match design knowledge concerned by a designer from a design knowledge base based on the to-be-fed-back design intention; and according to a design knowledge matching result, recommending to a designer in sequence by utilizing a knowledge recommendation algorithm. According to the method, the man-machine cooperation efficiency of the conceptual design is effectively improved.

Description

Technical Field

[0001] The present invention relates to the technical field of human - machine interactive cognitive concept design, and in particular, to a human - machine interactive cognitive concept design method and device based on deep learning and knowledge management. Background Art

[0002] Concept design is the core stage of new product development. At this stage, design information is incomplete, design iteration is the most complex, and design cognition is insufficient. Designers often rely on computer - aided concept design software to realize the visualization of design intentions. However, the existing computer - aided concept design software operation mode can only passively assist designers through geometric elements. There is a cognitive barrier between designers and the auxiliary design software. When encountering design stagnation, the existing auxiliary concept design software cannot recognize the designer's intention. Designers often need to seek external third - party design knowledge to help themselves continue the design, resulting in a decrease in design efficiency.

[0003] To address this problem, some existing research efforts are dedicated to changing the existing computer - aided concept design interaction mode. Niu et al. aimed to narrow the gap between humans and computer systems and designed a new auxiliary concept design software interaction interface that is more in line with natural human behavior. Hu et al. combined virtual reality technology with product appearance concept design. Designers can modify design parameters and view the appearance in a virtual environment until they determine the appearance and then exit the virtual environment and feedback the concept design data to the auxiliary design system. Wang proposed a concept design image enhancement algorithm based on deep belief networks and fed the enhanced design image back to designers to design more artistic images.

[0004] However, product concept design is a process highly dependent on designers' own experience and knowledge accumulation. These studies do not make up for the defect of insufficient design information in concept design. Therefore, for the problem of incomplete concept design information, cognitive algorithms based on deep learning and design knowledge management are regarded as an effective solution. Gao et al. predicted the designer's next concept design operation based on the design sequence in the process of architectural sketch concept design using a standard Transformer model. Raina et al. constructed a deep - learning framework with an encoder - decoder structure and combined it with an inference algorithm to construct an artificial concept design agent, realizing the modeling and analysis of the concept design sequence of ordinary trusses. Wang et al. serialized concept design knowledge using a Markov logic network and added time information to ensure the accuracy of design knowledge when matching with designers. Xia et al. used the theory of design knowledge management to represent concept design knowledge as a design knowledge graph with three characteristics: entity, relationship, and attribute, and then extracted keywords from the graph and the user - input natural language for design knowledge retrieval and matching.

[0005] When dealing with the cognitive barrier problem between designers and existing auxiliary design software, some scholars use conventional time series analysis algorithms to analyze the conceptual design operations of designers. These methods only learn the operation habits of designers and do not pay attention to the design attention of designers. There are also some scholars who use keyword matching to design knowledge for designers, and the pushed design knowledge has great ambiguity, low task relevance and low knowledge availability. And these methods are all one-way conceptual design information transmission in the conceptual design scenario, and the research goal is often one of "from designer to computer" and "from computer to designer", and the result still has the mutual cognitive barrier between designers and computers, which leads to its inability to effectively assist the conceptual design process. Summary of the Invention

[0006] The present invention provides a human-computer mutual cognitive conceptual design method based on deep learning and knowledge management, which effectively improves the efficiency of human-computer cooperation in conceptual design.

[0007] The technical solution of the present invention is as follows:

[0008] A human-computer mutual cognitive conceptual design method based on deep learning and knowledge management, comprising the following steps:

[0009] Step S1, collecting operation data in the historical conceptual design process of the designer to obtain design images and design descriptions;

[0010] Step S2, annotating the design intentions of the design images and design descriptions, mapping them to design intentions, and obtaining a private design multi-modal data set of the designer; using the design multi-modal data set to train a deep learning network to obtain an identification model for identifying the design intentions of the designer;

[0011] Step S3, collecting operation data in the current conceptual design process of the designer to obtain current design images and current design descriptions; using the identification model to identify based on the current design images and current design descriptions to obtain the design intentions to be fed back; using a design knowledge matching framework to match the design knowledge concerned by the designer from a design knowledge base based on the design intentions to be fed back;

[0012] Step S4, recommending to the designer in sequence according to the design knowledge matching result by using a knowledge recommendation algorithm.

[0013] Preferably, step S1 includes:

[0014] Step S101, monitoring the conceptual design sketch modeling process to obtain a design record video; extracting key design operations in the design record video and removing irrelevant operations to obtain design images;

[0015] Step S102, using oral description analysis for the design images to obtain design descriptions;

[0016] Step S103: Classify and extract the typical design intents of designers during the conceptual design process; map the design images and design descriptions to specific design intent categories.

[0017] In step S101, the key design operations include: adding a straight line, adding a ray, adding a circle with a fixed center, adding a tangent circle, adding a dotted center line, adding a reference point, adding a reference plane, adding geometric dimensions to a straight line, trimming a line segment, trimming an arc, modifying a right angle to a rounded corner, feature stretching, feature cutting, feature mirroring, adding an assembly part to the assembly space, setting the centers of two parts with circular end faces to be the same, setting two certain planes of two parts to coincide, setting a certain point to pass through a plane of another part, setting the distance between two certain planes of two parts to a fixed value, and setting the angle between two certain planes of two parts to a fixed value.

[0018] In step S101, the irrelevant operations include adjusting the operation perspective, switching the operation object, etc.

[0019] Step S102 includes: The designer makes an oral description based on the design operation image, and at the same time records the description process to obtain the oral description audio; transcribe the oral description audio into text, modify the non-standard language in the description, and supplement the language at necessary positions to obtain the design description.

[0020] In step S103, the typical design intents include adding linear design elements, adding circular design elements, adding reference design elements, dimension annotation, modifying the sketch, entity feature adjustment, adding parts, adding concentric fit, adding coincidence fit, and adding fixed fit.

[0021] The private design multi-modal dataset is divided into two parts: a training dataset and a test dataset. The training dataset is used to train the deep learning network, and the test dataset is used to obtain the design intents to be knowledge-fed back.

[0022] In step S2, the deep learning network is a multi-modal Transformer model based on T2T-ViT and Bert; using the training dataset to train the deep learning network includes:

[0023] Use the pre-trained T2T-ViT model and Bert model to vectorize the design images and design descriptions in the training dataset respectively, so that the deep learning network can recognize the design data information;

[0024] Use positional encoding embedding for the vectorized design data, so that the design data is embedded with spatio-temporal information;

[0025] Train a multi-modal Transformer model using the embedded data to obtain the pre-trained weights of the model.

[0026] When training the deep learning network using the training data set, the optimizer used is Adam, the loss function uses the cross-entropy algorithm, the learning rate is 0.001, and the neuron inactivation ratio is 0.1.

[0027] In step S3, use the design knowledge matching framework to match the design knowledge concerned by the designer from the design knowledge base based on the design intention to be fed back, including:

[0028] Step S301, obtain the design knowledge related to the conceptual design task and obtain the category of the design knowledge; obtain the emphasis information of the design knowledge, and the emphasis information includes the knowledge emphasis category and the knowledge attention matrix;

[0029] The design intention to be fed back includes the design intention category and the design attention matrix;

[0030] Use knowledge similarity calculation to match the associated design knowledge of the design intention to be fed back from the conceptual design knowledge base; the knowledge similarity calculation formula is as follows:

[0031]

[0032] Among them, Z k represents the matching score between the design intention and the design knowledge; ω1 and ω2 are the weights of the category matching score and the attention matrix matching score respectively, and it is set that ω1 + ω2 = 1 and 0 ≤ ω1, ω2 ≤ 1; represents the matching score between the design intention category and the emphasis category of the design knowledge; represents the matching score between the design intention attention matrix and the design knowledge attention matrix; k represents the knowledge category; represents the emphasis category of the design knowledge; represents the attention matrix of the design knowledge; DI Al represents the design intention category; DI Am represents the design attention matrix; f s (·) is used to judge the consistency between the design intention category and the emphasis category of the design knowledge, and returns 1 when the categories are consistent and 0 when the categories are inconsistent; f cs (·) represents using the cosine similarity to calculate the similarity between the attention matrix of the design intention and the attention matrix of the design knowledge.

[0033] The design knowledge comes from the design guidelines, design axioms, and historical design materials related to the design task; the design knowledge categories include expert design knowledge EDK and scenario design knowledge SDK related to the design task.

[0034] In the knowledge similarity calculation formula, when k = S, it means that the relevant result is the calculation result of the scenario design knowledge, and when k = E, it means that the relevant result is the calculation result of the expert design knowledge.

[0035] Step S4 includes:

[0036] S401: The designer determines whether the scenario design knowledge SDK or the expert design knowledge EDK is needed. If the EDK is needed, step S402 is executed; if the SDK is needed, it jumps to step S405;

[0037] S402: Recommend the expert design knowledge EDK to the designer according to the similarity matching score ranking of the expert design knowledge EDK;

[0038] S403: The designer determines whether the expert design knowledge EDK is needed again. If so, it jumps to step S402; if not, step S404 is executed;

[0039] S404: Determine whether the scenario design knowledge SDK is needed. If it is needed, step S405 is executed; if not, it jumps to step S408;

[0040] S405: Recommend the scenario design knowledge SDK to the designer according to the similarity matching score ranking of the scenario design knowledge SDK;

[0041] S406: The designer determines whether the scenario design knowledge SDK is needed again. If so, it jumps to step S405; if not, step S407 is executed;

[0042] S407: Determine whether the expert design knowledge EDK is needed. If it is needed, it jumps to step S402; if not, step S408 is executed;

[0043] S408: End the knowledge recommendation.

[0044] The present invention also provides a human-computer interactive cognitive concept design method device based on deep learning and knowledge management, including:

[0045] A data acquisition module that acquires the operation data in the historical concept design process of the designer, obtains the design image and design description, and maps them to the design intention;

[0046] A model training module that manually annotates the design image and design description to obtain the designer's private design multi-modal data set; divides the private design multi-modal data set into a training data set and a test data set; uses the training data set to train the deep learning network to obtain the model pre-training weights of the recognition model for identifying the designer's design intention;

[0047] The design knowledge matching module calls the pre-trained weights of the model, identifies the design intent of the test set through the recognition model, and obtains the design intent to be fed back; uses the design knowledge matching framework to match the design knowledge concerned by the designer from the design knowledge base based on the design intent to be fed back;

[0048] The design knowledge recommendation module recommends to the designer in sequence according to the design knowledge matching result by using the knowledge recommendation algorithm.

[0049] The method of the present invention first records the design operations of the designer during the conceptual design process to obtain the operation record video of the conceptual design, then extracts the key conceptual design operations from the video to obtain the design operation images; then uses the operation images to design the description text; establishes the mapping relationship between the design operations and the design intent, and performs manual annotation on the design operation images and the design descriptions; uses the pre-trained T2T-ViT model and Bert model to vectorize the training data, and then embeds the position encoding; then uses the multi-modal Transformer to model and learn the features of the conceptual design operations, and identifies the design intent of the designer; then based on the identified design intent, uses the design knowledge matching framework to match the conceptual design knowledge concerned by the designer from the conceptual design knowledge base; finally uses the knowledge recommendation algorithm to recommend the design knowledge to the designer based on the conceptual design knowledge matching result.

[0050] Compared with the prior art, the beneficial effects of the present invention are:

[0051] Aiming at the cognitive barrier problem between the designer and the computer in the conceptual design, the present invention proposes a human-computer mutual cognitive conceptual design method based on a deep learning model and conceptual design knowledge recommendation. The conceptual design method proposed by the present invention has feasibility and high efficiency. The present invention solves the human-computer cognitive barrier problem caused by incomplete information and complex iteration in the conceptual design process, and effectively improves the efficiency of human-computer collaboration in the conceptual design. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 is the method architecture diagram of the present invention;

[0053] Figure 2 is the structural diagram of the conceptual design intent recognition algorithm;

[0054] Figure 3 is the structural diagram of the design knowledge matching framework;

[0055] Figure 4 is the execution logic diagram of the knowledge recommendation algorithm; DETAILED DESCRIPTION OF THE INVENTION

[0056] The present invention will be further described in detail below in conjunction with the drawings and embodiments. It should be noted that the following embodiments are intended to facilitate the understanding of the present invention, but do not limit it in any way.

[0057] like Figure 1 The method architecture diagram of the present invention is shown, which includes the following steps:

[0058] Step S1: collect the operation data in the conceptual design process, obtain the design image and design description, and extract the key design intention in the conceptual design process;

[0059] S1 is specifically:

[0060] Step S101, monitor the conceptual design sketch modeling process to obtain a design operation record video; then, extract the key design steps that can reflect the designer's intention in the video, eliminate irrelevant designer operations, such as adjusting the operation perspective, switching the operation object, etc., and finally obtain the key design operation image.

[0061] Step S102, using spoken description analysis on the design image, the designer makes a spoken description based on the design operation image, and records the description process to obtain spoken description audio; then, the audio is transcribed into text, and the non-standard language in the description is modified, and language is supplemented at necessary locations to obtain key design operation descriptions.

[0062] Step S103, classify and extract the typical design intentions of the designer in the conceptual design process; then map the design image and design description to specific design intention categories;

[0063] There are ten categories of typical design intentions extracted, namely, adding linear design elements, adding circular design elements, adding reference design elements, dimensioning, modifying sketches, adjusting entity features, adding parts, adding concentric fits, adding coincident fits, and adding fixed fits. The mapping scheme between design operations and design intentions is summarized in Table 1.

[0064] Table 1: Mapping of design operations to design intent

[0065]

[0066] Step S2: Based on the conceptual design intent categories shown in Table 1, manual annotation is performed to classify the design images and design descriptions into specific intents, respectively, to obtain a private design multimodal dataset; the private design multimodal dataset is divided into two parts, one of which is used as a training dataset; the deep learning network is trained using the training dataset to obtain a model pre-training weight for identifying the designer's design intent;

[0067] The private design multimodal dataset is divided into a training dataset and a test dataset in a ratio of 4:1. The test dataset is used to obtain the design intention to be fed back by knowledge.

[0068] Optimize using a multimodal Transformer model based on T2T-ViT and Bert on the training dataset, Figure 2 showing the structure of the intention recognition algorithm used;

[0069] Use the pre-trained T2T-ViT model and Bert model to vectorize the design images and design descriptions in the training dataset respectively, enabling the deep learning network to recognize design data information;

[0070] Use positional encoding embedding for the vectorized design data, enabling the design data to embed spatio-temporal information.

[0071] As Figure 2 shown in the structural diagram of the conceptual design intention recognition algorithm, based on the multimodal Transformer model of T2T-ViT and Bert, the optimizer used during training is Adam, the loss function uses the cross-entropy algorithm, the learning rate is 0.001, the neuron inactivation ratio is 0.1, and the deep learning platform Pytorch is used for training. After the model training is completed, traditional algorithms for processing single-modal data (MobilenetV3, MobilenetV2, Bert, and Bert_RNN) are selected for performance comparison with the multimodal conceptual design data-driven intention recognition algorithm in this paper. The performance summary of the conceptual design intention recognition is shown in Table 2.

[0072] Table 2: Performance summary of the conceptual design intention recognition algorithm

[0073]

[0074] In step S3, call the pre-trained model weights to perform design intention recognition on another part of the private design multimodal dataset to obtain the design intention to be fed back; use the design knowledge matching framework to match the design knowledge concerned by the designer from the design knowledge base based on the design intention to be fed back;

[0075] The design intention to be fed back consists of the design intention category DI Al and the design attention matrix DI Am in two parts. DI Al reflects the designer's comprehensive perception of the current design state, while DI Am is a matrix containing the designer's conceptual design information output by the algorithm. It reflects the designer's current design focus and implicitly implies the key directions for activating the design memory in the current design stage.

[0076] Figure 3 For the structural diagram of the design knowledge matching framework, combined with Figure 1 、 Figure 3 , S3 is specifically as follows:

[0077] Step S301, Knowledge collection, to obtain the design knowledge related to the conceptual design task;

[0078] All conceptual design knowledge is sourced from design guidelines, design axioms, and historical design materials related to the design task.

[0079] Step S302, Knowledge registration, to obtain the categories of design knowledge;

[0080] The design knowledge categories consist of expert design knowledge (EDK) and scenario design knowledge (SDK) related to the design task.

[0081] Step S303, Knowledge extraction, to obtain the emphasis information of design knowledge using a pre-trained single-modal analysis model;

[0082] The emphasis information of each design knowledge consists of its knowledge emphasis category and knowledge attention matrix.

[0083] Step S304, Knowledge matching, to utilize similarity calculation to match the associated design knowledge of the design intent to be fed back;

[0084] The knowledge similarity calculation formula is as follows:

[0085]

[0086] Among them, Z k represents the matching score between the design intent and the design knowledge; ω1 and ω2 are respectively the weights of the category matching score and the attention matrix matching score, with ω1 + ω2 = 1 and 0 ≤ ω1, ω2 ≤ 1; represents the matching score between the design intent category and the emphasis category of the design knowledge; represents the matching score between the design attention matrix and the design knowledge attention matrix; k represents the knowledge category, when k = S, it indicates that the relevant result is the calculation result of scenario design knowledge, and when k = E, it indicates that the relevant result is the calculation result of expert design knowledge; represents the emphasis category of the design knowledge; represents the attention matrix of the design knowledge; f s (·) is used to judge the consistency between the design intent category and the emphasis category of the design knowledge, returning 1 when the categories are consistent and 0 when the categories are inconsistent; f cs (·) represents calculating the similarity between the attention matrix of the design intent and the attention matrix of the design knowledge using cosine similarity.

[0087] Step S4, Recommend to the designer in sequence according to the design knowledge matching result using the knowledge recommendation algorithm;

[0088] Figure 4 is the execution logic diagram of the knowledge recommendation algorithm, combined with Figure 1 、 Figure 3, Figure 4 , S4 specifically is as follows:

[0089] S401: Execute the knowledge recommendation algorithm. The designer determines whether the scenario design knowledge SDK or the expert design knowledge EDK is needed. If the EDK is needed, execute step S402. If the SDK is needed, jump to step S405.

[0090] S402: Recommend the EDK to the designer according to the sorted EDK similarity matching scores.

[0091] S403: The designer determines whether the EDK is needed again. If so, jump to step S402. If not, execute step S404.

[0092] S404: Determine whether the SDK is needed. If it is needed, execute step S405. If not, jump to step S408.

[0093] S405: Recommend the SDK to the designer according to the sorted SDK similarity matching scores.

[0094] S406: The designer determines whether the SDK is needed again. If so, jump to step S405. If not, execute step S407.

[0095] S407: Determine whether the EDK is needed. If it is needed, jump to step S402. If not, execute step S408.

[0096] S408: End the knowledge recommendation.

[0097] Nine indicators, namely knowledge availability, knowledge relevance, knowledge complexity, knowledge satisfaction, system reliability, system response speed, continuous response speed, interaction simplicity, and interaction participation, are selected to evaluate the performance of the conceptual design knowledge matching framework and the knowledge recommendation algorithm, and are summarized in Table 3.

[0098] Table 3: Summary of scores of five testers

[0099]

[0100] As can be seen from Table 2 and Table 3:

[0101] (1) In the computer's cognitive stage of the designer, that is, the conceptual design intention recognition part, the algorithm in this paper is superior to the traditional data analysis algorithm in all indicators, and the highest accuracy rate reaches 98.00%. In the comparison of indicators such as precision, recall rate, and F1score, the algorithm in this paper reaches 0.9802, 0.9817, and 0.9797 respectively.

[0102] (2) The cognitive stage of the designer towards the computer, i.e., the part of concept design knowledge recommendation. Five testers respectively experienced the knowledge recommendation process, and the average scores for each index exceeded 3.5, and the overall average score reached (3.89 + 4.11 + 3.78 + 3.89 + 4.11) / 5 = 3.96, proving that the method in this paper can effectively assist the designer in the concept design process.

[0103] So far, it shows that in the process of product concept design, the present invention can effectively realize the mutual cognition between the designer and the computer, and is conducive to the designer to carry out continuous design operations, avoiding the time-consuming search for third-party design knowledge, which is of great significance for improving the concept design efficiency.

[0104] The above-described embodiments have detailed the technical solutions and beneficial effects of the present invention. It should be understood that the above are only specific embodiments of the present invention and are not used to limit the present invention. Any modifications, supplements, equivalent replacements, etc. made within the scope of the principles of the present invention shall be included within the protection scope of the present invention.

Claims

1. A human-machine mutual cognition concept design method based on deep learning and knowledge management, characterized in that It includes the following steps: Step S1: Collect the operation data in the historical conceptual design process of the designer to obtain design images and design descriptions; Step S2: Annotate the design intentions of the design images and design descriptions, map them to design intentions, and obtain the private design multi-modal dataset of the designer; Use the design multi-modal dataset to train the deep learning network to obtain an identification model for identifying the design intentions of the designer; Step S3: Collect the operation data in the current conceptual design process of the designer to obtain the current design image and the current design description; Use the identification model to identify based on the current design image and the current design description to obtain the design intention to be fed back; Use the design knowledge matching framework to match the design knowledge concerned by the designer from the design knowledge base based on the design intention to be fed back; Step S4: Recommend to the designer in sequence according to the design knowledge matching result by using the knowledge recommendation algorithm.

2. The method for human-machine mutual cognitive concept design based on deep learning and knowledge management according to claim 1, wherein Step S1 includes: Step S101: Monitor the conceptual design sketch modeling process to obtain a design record video; extract the key design operations in the design record video, eliminate the irrelevant operations, and obtain design images; Step S102: Use oral description analysis for the design images to obtain design descriptions; Step S103: Classify and extract the typical design intentions of the designer in the conceptual design process; map the design images and design descriptions to specific design intention categories.

3. The human-machine mutual cognitive concept design method based on deep learning and knowledge management according to claim 2, wherein In Step S101, the key design operations include: adding a straight line, adding a ray, adding a circle with a fixed center, adding a tangent circle, adding a center line by point drawing, adding a reference point, adding a reference plane, adding geometric dimensions to a straight line, trimming a line segment, trimming an arc, modifying a right angle to a fillet, feature stretching, feature cutting, feature mirroring, adding an assembly part to the assembly space, setting the centers of two parts with circular end faces to be the same, setting two certain planes of two parts to coincide, setting a certain point to pass through a plane of another part, setting the distance between two certain planes of two parts to a fixed value, and setting the angle between two certain planes of two parts to a fixed value.

4. The method for human-machine mutual cognitive concept design based on deep learning and knowledge management according to claim 2, wherein The typical design intentions include adding linear design elements, adding circular design elements, adding reference design elements, dimensioning, modifying the sketch, adjusting solid features, adding parts, adding concentric fits, adding coincidence fits, and adding fixed fits.

5. The human-machine mutual cognitive concept design method based on deep learning and knowledge management according to claim 1, characterized in that The deep learning network is a multi-modal Transformer model based on T2T-ViT and Bert; Using the training dataset to train the deep learning network includes: Use the pre-trained T2T-ViT model and Bert model to vectorize the design images and design descriptions in the training dataset respectively, so that the deep learning network can identify the design data information; Perform position encoding embedding on the vectorized design data so that the design data is embedded with spatio-temporal information; Use the embedded data to train the multi-modal Transformer model to obtain the pre-trained weights of the model.

6. The human-machine mutual cognitive concept design method based on deep learning and knowledge management according to claim 5, characterized in that When training a deep learning network using a training data set, the optimizer adopted is Adam, the loss function adopts the cross-entropy algorithm, the learning rate is 0.001, and the neuron inactivation ratio is 0.

1.

7. The human-machine mutual cognitive concept design method based on deep learning and knowledge management according to claim 1, characterized in that In step S3, a design knowledge matching framework is used to match design knowledge concerned by designers from a design knowledge base based on the design intent to be fed back, including: Step S301: Obtain design knowledge related to the conceptual design task and obtain the category of the design knowledge; obtain the focus information of the design knowledge, where the focus information includes a knowledge focus category and a knowledge attention matrix; The design intent to be fed back includes a design intent category and a design attention matrix; Use knowledge similarity calculation to match the associated design knowledge of the design intent to be fed back from the conceptual design knowledge base; the knowledge similarity calculation formula is as follows: Among them, Z k represents the matching score between the design intention and the design knowledge; ω1 and ω2 are the weights of the category matching score and the attention matrix matching score respectively, and it is set that ω1 + ω2 = 1 and 0 ≤ ω1, ω2 ≤ 1; represents the matching score of the emphasis category between the design intention category and the design knowledge; represents the matching score of the design intention attention matrix and the design knowledge attention matrix; k represents the knowledge category; represents the emphasis category of the design knowledge; represents the attention matrix of the design knowledge; DI Al represents the design intention category; DI Am represents the design attention matrix; f s (·) is used to judge the consistency between the design intention category and the emphasis category of the design knowledge, and returns 1 when the categories are consistent and 0 when the categories are inconsistent; f cs (·) represents using the cosine similarity to calculate the similarity between the attention matrix of the design intention and the attention matrix of the design knowledge.

8. The human-machine mutual cognitive concept design method based on deep learning and knowledge management according to claim 7, characterized in that The design knowledge comes from design guidelines, design axioms, and historical design materials related to the design task; The design knowledge category includes expert design knowledge EDK and scenario design knowledge SDK related to the design task.

9. The human-machine mutual cognitive concept design method based on deep learning and knowledge management according to claim 8, characterized in that Step S4 includes: S401: The designer judges whether scenario design knowledge SDK or expert design knowledge EDK is needed. If EDK is needed, step S402 is executed; if SDK is needed, jump to step S405; S402: Recommend expert design knowledge EDK to the designer according to the similarity matching score ranking of the expert design knowledge EDK; S403: The designer judges whether expert design knowledge EDK is needed again. If so, jump to step S402; if not, step S404 is executed; S404: Judge whether scenario design knowledge SDK is needed. If needed, step S405 is executed; if not, jump to step S408; S405: Recommend scenario design knowledge SDK to the designer according to the similarity matching score ranking of the scenario design knowledge SDK; S406: The designer judges whether scenario design knowledge SDK is needed again. If so, jump to step S405; if not, step S407 is executed; S407: Judge whether expert design knowledge EDK is needed. If needed, jump to step S402; if not, step S408 is executed; S408: End the knowledge recommendation.

10. A human-computer mutual cognitive conceptual design method device based on deep learning and knowledge management, including: A data acquisition module that acquires operation data in the designer's historical conceptual design process, obtains design images and design descriptions, and maps them to design intents; A model training module that annotates the design intents of the design images and design descriptions, maps them to design intents, and obtains the designer's private design multimodal data set; Use the design multimodal data set to train a deep learning network to obtain an identification model for identifying the designer's design intent; A design knowledge matching module that acquires operation data in the designer's current conceptual design process, obtains the current design image and the current design description; Use the identification model to perform identification based on the current design image and the current design description to obtain the design intent to be fed back; Use the design knowledge matching framework to match the design knowledge concerned by the designer from the design knowledge base based on the design intention to be feedback. The design knowledge recommendation module recommends to the designer in sequence according to the design knowledge matching result using the knowledge recommendation algorithm.

Images