AR-based industrial design review method, server, medium and program product

Abstract

The invention provides an AR-based industrial design review method, a server, a medium and a program product, and relates to the field of Internet. The method comprises the following steps: receiving a three-dimensional industrial design-containing review application sent by a user side; extracting data features of the three-dimensional industrial design, and matching the data features with design features stored in a preset database to obtain n similar designs with the highest matching degree; analyzing difference points between the three-dimensional industrial design and the n similar designs, and generating a preliminary review judgment report; after detecting a review acceptance request sent by the review end, sending the review acceptance request to the review end; receiving design review suggestions sent by the review end based on multiple modes; analyzing the design review suggestions through an image recognition and natural language processing technology, and generating a design review result; and sending the design review result to the user side. Accurate presentation of industrial design details and functions is achieved, effective data support and analysis means are provided for review, and review work of industrial design can be completed comprehensively and efficiently.

Description

Technical Field

[0001] The present application relates to the field of the Internet, and in particular to an AR-based industrial design review method, server, medium and program product. Background Art

[0002] Against the backdrop of global economic integration, competition in industrial design has become increasingly fierce, and companies are placing higher demands on the scientificity and efficiency of design reviews.

[0003] At present, many companies mainly rely on traditional review tools and methods when conducting industrial design reviews. For example, using simple drawing software to display design plans, or evaluating through physical models, reviewers rely on their own perspectives and experience to judge the pros and cons of the design.

[0004] However, this approach has many shortcomings. On the one hand, traditional review tools and methods are difficult to accurately present the details and functions of the design, which easily leads to deviations in the review results, thus affecting the scientificity and fairness of the review. Therefore, it is difficult for existing methods to comprehensively and efficiently review industrial designs. Summary of the invention

[0005] The present application provides an AR-based industrial design review method, server, medium and program product, which are used to achieve comprehensive and efficient completion of industrial design review work when the existing technology has difficulty in accurately displaying design details and lacks effective analysis and evaluation methods.

[0006] In a first aspect, the present application provides an AR-based industrial design review method, which is applied to a server of a review system, and the method includes: receiving a review application containing a three-dimensional industrial design sent by a user terminal; extracting data features of the three-dimensional industrial design, and matching them with design features stored in a preset database to obtain n similar designs with the highest matching degree, where n is a positive integer set in advance; based on a preset preliminary design review model, analyzing the differences between the three-dimensional industrial design and the n similar designs, and generating a preliminary review judgment report, wherein the preliminary design review model is a model established based on training based on historical design review data; after detecting a review acceptance request sent by the review terminal, sending the preliminary review judgment report to the review terminal; receiving design review opinions sent by the review terminal in a variety of ways; parsing the design review opinions through image recognition and natural language processing technology to generate design review results; and sending the design review results to the user terminal.

[0007] By adopting the above-mentioned technical solution, accurate presentation of industrial design details and functions is achieved, effective data support and analysis means are provided for the review, quantitative evaluation of the design can be carried out, the scientificity and fairness of the review are improved, and the review of industrial design can be completed comprehensively and efficiently.

[0008] In combination with some embodiments of the first aspect, in some embodiments, after detecting the review acceptance request sent by the review end, the step of sending the preliminary review judgment report to the review end specifically includes: receiving the review request containing reviewer information sent by the review end; after querying the historical review database corresponding to the reviewer information, obtaining the reviewer's historical review preferences; according to the historical review preferences, extracting corresponding content and precautions from the preliminary review judgment report to generate personalized review suggestions; and sending the personalized review suggestions to the review end.

[0009] By adopting the above technical solutions, reviewers can pay more attention to the content related to their preferences, improve the pertinence and efficiency of the review. At the same time, personalized review suggestions can better meet the needs of reviewers and improve the review experience.

[0010] In combination with some embodiments of the first aspect, in some embodiments, after the step of extracting the data features of the three-dimensional industrial design and matching them with the design features stored in a preset database, it also includes: extracting the design review opinions, the three-dimensional industrial design and the corresponding product structure data and marking them as different types of entity nodes, and inserting them into a preset industrial design knowledge graph; establishing the relationship between the entity nodes in the industrial design knowledge graph, and determining the correlation between different corresponding nodes of the three-dimensional industrial design under the same product category through the reasoning algorithm of the knowledge graph; determining an existing design scheme that has a high correlation with the user's current three-dimensional design scheme based on the correlation; determining a design optimization scheme based on the existing design scheme, and the design optimization scheme is used to guide the user to improve the current three-dimensional industrial design based on the existing design scheme.

[0011] By adopting the above technical solutions, users can learn from the advantages of existing design solutions, improve the current design, and improve the design quality. At the same time, the application of knowledge graphs can integrate and mine design-related information to provide support for design innovation.

[0012] In combination with some embodiments of the first aspect, in some embodiments, after the step of receiving the design review opinions sent by the review end based on a variety of methods, it also includes: performing a format check on the design review opinions to determine the review opinion type, which includes text, voice, image, video and motion trajectory; according to the review opinion type, entering the design review opinions in a corresponding format.

[0013] By adopting the above technical solution, the accurate input of review opinions can be ensured, which is convenient for subsequent processing and analysis. Different types of review opinions can more comprehensively reflect the reviewer's opinions and ideas, and improve the comprehensiveness and accuracy of the review.

[0014] In combination with some embodiments of the first aspect, in some embodiments, after the step of entering the design review opinion in a corresponding format according to the review opinion type, the step also includes: extracting keywords of the received design review opinion; retrieving the entered design review opinions containing the keywords in a design review opinion database; calculating the similarity between the received design review opinion and the retrieved entered design review opinion; if the similarity is higher than a preset threshold, automatically classifying the received design review opinion under the corresponding entered design review opinion item.

[0015] By adopting the above technical solution, the review opinions can be automatically classified and sorted, and the efficiency of the review work can be improved. At the same time, similar review opinions can be found through similarity calculation, providing more targeted suggestions for design improvement.

[0016] In combination with some embodiments of the first aspect, in some embodiments, before the step of receiving design review opinions sent by the review end based on multiple methods, it also includes: receiving reviewer identification information from the review end; determining the review authority of the corresponding reviewer based on the reviewer identification information; if the review authority meets the review requirements, receiving and entering the corresponding review opinions; if the review authority does not meet the review requirements, sending a prompt message of insufficient review authority to the review end.

[0017] By adopting the above technical solution, the security and standardization of the review process can be ensured to a certain extent. Only reviewers with corresponding review authority can participate in the review, ensuring the reliability of the review results.

[0018] In combination with some embodiments of the first aspect, in some embodiments, the step of receiving design review opinions sent by the review end based on multiple methods specifically includes: controlling the virtual reality device to load the virtual model of the three-dimensional industrial design; when receiving a gesture operation from the reviewer of the review end, identifying the corresponding gesture trajectory and controlling the virtual model to render the gesture trajectory at the corresponding position; when detecting that the reviewer from the review end performs voice input, performing voice recognition to obtain voice text information and controlling the virtual model to render the voice annotation at the corresponding position; when detecting that the reviewer from the review end performs an image acquisition operation, obtaining the uploaded image and controlling the virtual model to render the image annotation at the corresponding position; determining the design review opinion according to the gesture trajectory, the voice annotation, and the image annotation of each reviewer.

[0019] By adopting the above technical solutions, a more intuitive and rich review method can be provided. Reviewers can express their opinions in a variety of ways, making the review more comprehensive and accurate. At the same time, the application of virtual reality technology can enhance the immersion and interactivity of the review and improve the efficiency and quality of the review.

[0020] In a second aspect, the present application provides a server, comprising: one or more processors and a memory; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code includes computer instructions, and the one or more processors call the computer instructions to cause the server to execute the method described in the first aspect and any possible implementation of the first aspect.

[0021] In a third aspect, the present application provides a computer-readable storage medium comprising instructions, which, when executed on a server, causes the server to execute the method described in the first aspect and any possible implementation of the first aspect.

[0022] In a fourth aspect, the present application provides a computer program product, which, when executed on a server, enables the server to execute the method described in the first aspect and any possible implementation manner of the first aspect.

[0023] One or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages: 1. The technical means of extracting the data features of 3D industrial design and matching them with the design features stored in the preset database, and analyzing the differences based on the preset design preliminary review model, effectively solves the problem of accurate presentation of design details and functions in the existing technology, resulting in deviations in the review results, thereby achieving a rapid and accurate evaluation of 3D industrial design, and providing a reliable foundation for subsequent in-depth reviews.

[0024] 2. Due to the adoption of a technical means of receiving the review request containing the reviewer information sent by the review end, querying the historical review database to obtain the reviewer's historical review preferences, and generating personalized review suggestions based on this, it effectively solves the problem of lack of specificity and low efficiency of the review in the existing technology, and thus achieves the technical effect of meeting the personalized needs of the reviewers and improving the review efficiency and experience.

[0025] 3. Since the reviewer identification information of the review receiving end is used to determine the review authority, and whether to accept the input review opinions is decided based on the authority, the problem of insufficient security and standardization of the review process in the existing technology is effectively solved, thereby achieving the technical effect of ensuring the reliability of the review results and ensuring the compliance of the review process. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 It is a flowchart of the industrial design review method based on AR in the embodiment of the present application; Figure 2 is another flowchart of the industrial design review method based on AR in an embodiment of the present application; Figure 3 It is a schematic diagram of a physical device structure of a server in an embodiment of the present application. DETAILED DESCRIPTION

[0027] The terms used in the following embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to be used as limitations to the present application. As used in the specification and appended claims of the present application, the singular expressions "one", "a kind of", "said", "above", "the" and "this" are intended to also include plural expressions, unless there is a clear indication to the contrary in the context. It should also be understood that the term "and / or" used in the present application refers to and includes any or all possible combinations of one or more listed items.

[0028] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as suggesting or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features, and in the description of the embodiments of the present application, unless otherwise specified, "plurality" means two or more.

[0029] For ease of understanding, an application scenario of the relevant method is introduced below.

[0030] Suppose an automobile manufacturer is conducting an industrial design review for a new sedan. In the review room, the design team is nervously waiting for the review results. In the traditional review method, designers use simple drawing software to present the exterior and interior design of the vehicle. However, these two-dimensional drawings cannot accurately present the smoothness of the vehicle's lines, the delicate details of the door handles, and the ergonomic design of the seats. The reviewers can only squint their eyes and guess the actual effect with limited experience and imagination. Then, the physical model is pushed in. However, due to the difference in the proportion of the model and the real vehicle, some complex structures in the car, such as the layout of the center console and the display function of the instrument panel, are difficult to show clearly. The reviewers circled around the model, but still found it difficult to fully understand the pros and cons of the design.

[0031] In related technologies, industrial design is limited to two-dimensional drawings or simulations, which makes it difficult to fully display the details and three-dimensional sense of the design model, and easily leads to deviations in the evaluation results.

[0032] The AR-based industrial design review method in the embodiment of the present application extracts the data features of the three-dimensional industrial design and matches them with a preset database, and generates a preliminary review judgment report based on the preset model analysis points, thereby achieving a comprehensive and accurate evaluation of the industrial design. It can not only effectively present the details and functions of the design, but also provide effective data support and analysis methods for the reviewers.

[0033] For ease of understanding, the following describes the process of the method provided by this implementation in combination with the above scenario. Figure 1 , which is a flow chart of an AR-based industrial design review method in an embodiment of the present application.

[0034] S101, receiving a review application containing a three-dimensional industrial design sent by a user terminal; The server first needs to establish a connection with the client device, such as communicating with the server through a mini program or APP. The client device can be a mobile phone or other device used by the designer. The server will start the network service and listen for connection requests from the client. When the client sends a connection request, the server will respond to the request and establish a connection with the client.

[0035] After the connection is established, the server will receive the review application file sent by the user. The review application file contains complete three-dimensional industrial design data and usually adopts standard three-dimensional model file formats, such as STL, OBJ, 3DS, etc. The server needs to include a parsing module for the corresponding three-dimensional model file, which can load and parse these three-dimensional model files to restore the complete three-dimensional industrial design. For example, for an STL file, the server will parse all the triangular face data contained in the file and stitch them into a complete three-dimensional model through triangular facets. After the server parses the three-dimensional model file, it needs to optimize the model, including model repair, mesh optimization, etc., to ensure that subsequent analysis and processing can be based on the intact three-dimensional model. In addition, the server also needs to detect the rationality of the three-dimensional model. If the model has serious defects that make it difficult to complete subsequent work, the server needs to feedback the problem to the user and request the three-dimensional model file to be re-provided.

[0036] S102, extracting data features of the three-dimensional industrial design and matching them with design features stored in a preset database to obtain n similar designs with the highest matching degree, where n is a positive integer set in advance; After the server obtains the complete 3D industrial design model, it needs to analyze and extract its features. These design features can include extracting the overall structural parameters of the product, such as length, width, height, volume, surface area, etc.; it can also extract the overall contour curve as a feature. Detect and analyze the details of the product and extract structural features such as bolt holes and protrusions, which can be achieved with the help of computer vision technology. Analyze the surface material of the industrial design product and extract material parameters such as color, texture, reflection characteristics, etc. Analyze the components of the industrial design and their assembly methods, and extract the logical relationship and connection method between the components.

[0037] After extracting these design features, the server needs to match them with the design feature database, which stores a large number of design features of different products. The server will calculate the matching degree between products based on the parameter distance of the design features, and return the n similar designs with the highest matching degree to the current 3D design. The matching degree can be calculated using the vector cosine similarity algorithm. The returned similar design results will provide a reference for subsequent review and analysis. The setting of the n value will affect the breadth of the reference results. It is generally set reasonably according to the size of the database and should not be too large or too small.

[0038] In some embodiments, the server can extract text data from the design review opinions, as well as the corresponding digital model and product structure data of the three-dimensional industrial design. These data can be regarded as different types of entities, such as "body design" and "seat" as product structure entities, "curvature" and "aerodynamics" as design review attribute entities, etc. Then, the server will insert these entities into a preset industrial design knowledge graph as nodes. The knowledge graph is organized by product category and can reflect the association between product structures under a category. When inserting a new node, the server will automatically mark the type of each node, such as "product", "component", "parameter", etc. After inserting the node, the server will further establish the relationship between the nodes in the knowledge graph, such as establishing an "attribute" relationship between the "body design" node and the "curvature" node. This can be achieved by analyzing the node content and semantics. The key is to determine the degree of association between different design solution nodes of the same category of products.

[0039] The server will call the reasoning algorithm of the knowledge graph to calculate the correlation between design solutions based on the relationship between nodes. For example, calculate the correlation score between "Car A Design" and "Car B Design". If the correlation path between the product structure nodes and design parameter nodes of the two design solutions in the graph is shorter, the correlation is higher. Finally, the server can find the existing design solutions with the highest correlation with the user's current design solution in the knowledge graph. Because these recommended design solutions have a high similarity with the user's current solution, their successful design experience can be used to guide users to optimize their designs.

[0040] For example, the server can analyze the differences in design parameter selection among related design schemes and determine design optimization suggestions based on the differences, such as prompting users to modify the body curve radius, adjust the seat material, etc. to make design improvements.

[0041] S103, based on a preset preliminary design review model, analyzing the differences between the three-dimensional industrial design and the n similar designs, and generating a preliminary review judgment report, wherein the preliminary design review model is a model established by training based on historical design review data; Specifically, the construction process of the design preliminary review model is as follows: the training data of the model includes two parts. One part is the three-dimensional model data of industrial design products. This part of data includes a large number of three-dimensional digital models of different industrial design products, covering various fields such as electrical appliances, furniture, machinery, etc. Each three-dimensional model contains complete design data, such as geometric structure, material, assembly relationship and other information. These digital three-dimensional models can accurately reflect the details of the design plan. The other part is the manual review result data of the corresponding design product. This part of data is the result of the review by experienced design review experts based on the three-dimensional model, including qualitative descriptions of the design pros and cons, design defect reports, design improvement suggestions, etc. This part of data reflects the judgment of professionals on the design quality. In the process of model training, the above two types of data will be combined, the three-dimensional model data will be used as the input of the model, and the manual review result data will be used as the prediction target and supervision signal of the model. By analyzing a large amount of training data, the model gradually learns the intrinsic connection between the associated design three-dimensional data and the evaluation results, establishes the prediction rules for evaluating the design quality, and finally forms a preset design preliminary review model with design evaluation capabilities. With each training of new data, the design evaluation capability of the model will be further improved to meet a wider range of design review needs.

[0042] The server needs to analyze the differences between the current three-dimensional industrial design and the n similar designs matched in step S102 based on the preset preliminary design review model, and generate a preliminary review and judgment report for the design. When conducting the current three-dimensional industrial design review, the server will first input the complete three-dimensional digital model of the design into the preliminary design review model. The model will automatically analyze the data of the three-dimensional model and predict a design review result as the preliminary review result of the design scheme. Then, the server will further extract the differences between the design and the n similar designs matched in step S102 in terms of structure, style, size, material, components, etc. This needs to be achieved through technical means such as three-dimensional model registration, shape analysis, and image processing. For example, the differences between the two designs in terms of body curve shape, front shape, etc. can be extracted; the structural differences of components such as seats and steering wheels can be detected; the differences in body color, window material, etc. can be analyzed.

[0043] Finally, the server will generate a preliminary review report based on the model evaluation results and design difference analysis. The report will comprehensively consider the advantages and disadvantages of the overall structure, style, manufacturing process, etc. of the design, such as the evaluation results of the smoothness of the body lines, the efficiency of internal space utilization, and the difficulty of manufacturing; at the same time, it will list the specific differences with similar designs. If there are design defects, they will also be pointed out in the report, and suggestions for improvement will be given. This preliminary review report can intuitively reflect the characteristics of the design plan, judge its overall quality, and lay the foundation for subsequent in-depth review.

[0044] S104, after detecting the review acceptance request sent by the review end, sending the preliminary review judgment report to the review end; After the above preliminary design review report is generated, the server will continue to monitor the status of the review end. The review end is usually a client device used by the design review expert. When the review is ready, the review end will send a formal request to accept the review to the server.

[0045] After receiving the review request sent by the review end, the server needs to first authenticate the request initiator to ensure that the request comes from an authorized reviewer. The specific steps are as follows: First, the server will receive the review request data packet sent by the review end containing the reviewer identification information. The reviewer identification information can be encoded in a digital certificate or other identity-identifying encoding method, such as the reviewer's name, the hash value of the encrypted work number, etc. After receiving the review request, the server will extract the reviewer identification information in the data packet and query the reviewer identity database to see whether the reviewer identity corresponding to the identification information is legal. The database stores the identity information and identification codes of all authorized reviewers. If there is a corresponding record in the database, it proves that the review request comes from an authorized person and the verification is passed. Otherwise, it is considered an unauthorized request. For the review request that has passed the verification, the server also needs to further determine whether the reviewer's review authority meets the requirements of the current design project. For example, a junior reviewer can only review products with simple structures. If the complexity of the requested project exceeds his authority, the server will return a prompt message to the requesting end, requiring a higher-level reviewer to conduct the review.

[0046] The server can query the historical review project authority data of the reviewer stored in the database to determine the scope of his review qualifications. If the authority meets the requirements, the server will allow him to participate in the review of this design project, receive and record the review opinions submitted by him; if the authority is insufficient, the server will feedback a prompt message to the client of the review end, notifying that the review authority is insufficient and he cannot participate in this review. By confirming the identity and review authority of the review end, the server can ensure that only authorized review experts can evaluate the design plan, thereby ensuring the security and accuracy of the review process. After verification, the server will review the current design project to be reviewed and send the preliminary review judgment report of the project to the requesting review end. The preliminary review report will include a text description of the design plan, a three-dimensional digital model, and the design evaluation results generated by the server based on the automatic judgment model, including brief comments on product structure, shape, process, etc.

[0047] The server can actively push the review report file to the review end through a secure network connection, or wait for the review end to send a command to obtain the report, and then return the corresponding review report file data. The presentation of design information in the report can also be adjusted according to the needs of the review end, such as providing independent documents, or generating web reports, etc., to achieve personalized customization of the review judgment report and facilitate the work of the review end.

[0048] By sending the automatically generated preliminary design review report to the review end, the review experts can quickly understand the characteristics and possible problems of the design solution, and conduct in-depth design review work based on this, avoiding duplication of work and improving review efficiency.

[0049] In some embodiments, when the server receives the review request from the review end, it will extract the reviewer identity information in the request data packet, such as the reviewer's name, work number, etc. Then, the server needs to query the historical review database based on the reviewer information, which stores the historical review record data of all reviewers. The server will retrieve all the historical review cases of the reviewer, analyze his / her review focus, comment characteristics, etc. in different cases, and form the review preferences of the reviewer. For example, through analysis, it is found that a reviewer pays more attention to the ergonomics and human-computer interaction of industrial design solutions in past cases, and the proposed modification opinions are often concentrated in these aspects. Then, after obtaining the reviewer's historical review preferences, the server will extract the content and precautions that the reviewer may be interested in from the preliminary review report of the current design solution based on these preferences, and form personalized review suggestions. For example, determine the content of the human-computer interaction of the design solution from the report, increase the number of comments in this area; in the report precautions, put ergonomics-related matters in front, etc. Finally, the server will send the generated personalized review suggestions to the requesting review end for the reviewer's reference.

[0050] Through the above embodiments, compared with a unified review report, personalized suggestions can help reviewers lock in their areas of concern more quickly and provide a more personalized review experience. At the same time, personalized suggestions also make the review more targeted. Reviewers can focus on their areas of expertise and give full play to their professional advantages, rather than just trying to get by, thereby improving the efficiency and quality of the review. In summary, the server can intelligently analyze historical data, gain an in-depth understanding of the expertise preferences of each review expert, and generate personalized review suggestions based on this, to achieve a more intimate and efficient review process and enhance the user experience.

[0051] S105, receiving design review opinions sent by the review end in various ways; After sending the preliminary design review report to the review end, the server needs to continue to wait for and receive the design review comments sent by the review end. Considering that the review comments may be fed back in multiple formats, the server needs to support receiving different types of opinion data including text, voice, pictures, videos, and gesture operation trajectories.

[0052] When the server receives the review opinions from the review client, it needs to distinguish the data of different types of opinions and process them accordingly. The specific steps are as follows: First, the server will control the connected virtual reality device to load the virtual model of the three-dimensional industrial design for the reviewer to browse and interact with. When the server detects that the reviewer on the review side performs gesture operations on the virtual model, such as finger tapping or palm grasping in a virtual space of a certain part of the model, the server will capture these gesture trajectories in real time and analyze the gesture path, direction, and force parameters through the gesture recognition algorithm to determine the reviewer's operation intention. For example, it is judged that a circle is drawn on the model door, indicating that the part needs attention or has problems. After knowing the gesture intention, the server will control the virtual model to render the gesture trajectory at the corresponding door part position, that is, the position will display the trajectory as a mark of the review opinion. Secondly, when the server detects that the reviewer on the review side performs voice input, such as voice description or comment on the model, the server will start the voice recognition module, obtain the voice data in real time and convert it into text information, and perform semantic analysis at the same time to obtain the review opinion content to be expressed by the voice. Then, the server will render the voice text at the relevant position of the virtual model as a mark of the voice review opinion. Thirdly, if the reviewer is detected to be taking a photo or recording the screen, the server will obtain the pictures or videos uploaded by the reviewer and analyze their content using image / video processing technology, such as detecting the objects or scenes that appear. Finally, the server will render these pictures / videos at the appropriate virtual model location to complete the visual annotation of the opinions.

[0053] In summary, the server can distinguish various data types of review opinions, and adopt technical means such as gesture trajectory recognition, voice recognition, image / video analysis, etc. for different types to parse the content of the opinions and render them on the virtual model to form interactive visual design review opinions for the server to comprehensively analyze and generate text review reports. By supporting multimodal design review opinions, the review end can express its views on the design more intuitively and accurately, and enhance the interactive experience. The server can fully understand and analyze these opinions to make the final review results more comprehensive and reliable. This technical means effectively combines virtual simulation and interactive functions, providing important support for AR-based industrial design reviews.

[0054] After receiving review comments in different formats, the server needs to perform preprocessing to facilitate subsequent parsing.

[0055] In some embodiments, the server can set weights for different reviewers so that when comprehensively considering the opinions of each reviewer, it can distinguish their importance and give different influences. The weight of the reviewer can be set according to the following aspects: (1) The professional qualification level of the reviewer. More senior review experts have higher weights. (2) The professional field of the reviewer. Reviewers who are more professional in a certain field have higher weights in that field. (3) The reviewer's historical review record. Experts with higher review effectiveness have higher weights. (4) The reviewer's familiarity with the current product. Experts who are more familiar with the product have higher weights. The server can set a weight value between 0 and 1 for each reviewer. The more important the opinion, the higher the weight of the reviewer. A threshold weight value can also be set. Only the opinions of reviewers exceeding the threshold value will be adopted.

[0056] When generating a design review report, the server will consider the weight of each expert and comprehensively calculate the review results and design scores. The higher the weight of the expert opinion, the greater the impact on the final review conclusion. This allows the review opinions with higher professionalism and representativeness to play a leading role, and the review results are more fair and reliable.

[0057] S106. Analyze the design review opinions through image recognition and natural language processing technology to generate design review results; After pre-processing various review opinions, the server needs to use technologies such as image recognition and natural language processing to parse the specific content of the opinions and generate the review results of the design product.

[0058] Specifically, for text opinions, named entity recognition can be used to extract the names of product components contained in the opinion text; dependency syntax analysis can be used to obtain the evaluation objects and evaluation words in the opinions. For the text converted from voice opinions, the same natural language processing can be performed to obtain the semantic content of the opinions. For image / video opinions, the objects or scenes that appear in the opinions can be identified through preset detection technology; and behavioral analysis can be used to understand the operational behaviors of the review end, such as pointing and drawing on the product model. For gesture trajectories, the direction and shape of the trajectory can be analyzed to determine its intention, such as circling a part to express concern or a question.

[0059] Based on the analysis results of various review opinions, the server can generate a detailed design review report, which will identify the problems in the design and suggestions for improvement, give comments from professionals, and also include some positive evaluations. The report must cover all aspects of the design plan.

[0060] S107, sending the design review result to the user end.

[0061] After the design review is completed, the server needs to send the generated design review result report back to the user who originally submitted the design review application, which is usually a product designer. During the sending process, the server needs to filter the content of the review report appropriately, remove some comments that may involve business secrets or negative impacts, and ensure the reliability of the report. At the same time, the server can consider the organizational structure and content expression of personalized reports, and make customized adjustments for different user ends. For example, for senior designers with rich design experience, a detailed professional review report can be provided; for junior designers, the design improvement ideas and suggestions in the report can be emphasized. The server can also group and filter the review results by department, product line, etc.

[0062] In addition, the server can also provide design score history statistics charts to help the user side intuitively understand and analyze the changing trends of design capabilities, evaluate and compare the design performance in different time periods, and find the rules of improvement or decline in design capabilities so as to make timely improvements. Among them, the design score history statistics chart is obtained by visually drawing the design score data and related information generated by the server after each design review. Designers on the user side can analyze the trend of design quality changes by observing the design score history chart, for example, the exterior design capabilities of cars are steadily improving.

[0063] After receiving the review report, users can fully understand the opinions of the expert reviewers, identify the defects and deficiencies in the design, and make targeted improvements in subsequent design iterations. The server can also generate a visual solution of the design modification suggestions in the report for reference by the user.

[0064] In the above embodiment, the industrial design review method based on augmented reality technology in the embodiment of the present application is adopted. By extracting the features of the three-dimensional digital model and matching it with the database, using the design review pre-training model for difference analysis, supporting multi-modal review opinion analysis and other technical means, the three-dimensional sense and details of the industrial design are accurately presented, which provides effective quantitative analysis and evaluation support for the design review, which can not only effectively display the functions and details of the design scheme, but also make the review process more scientific and standardized.

[0065] In some embodiments, an industrial design may be reviewed by multiple reviewers, and the review results of a reviewer may also involve multiple aspects. If the review results of an industrial design include review suggestions for various aspects and are not categorized and sent to the user, the user will need to spend energy and time to review them, which will to a certain extent reduce the designer's efficiency in improving the industrial design.

[0066] The following is a more detailed description of the process of the method provided by this implementation. Figure 2, which is another flow chart of the AR-based industrial design review method in an embodiment of the present application.

[0067] S201, performing a format check on the design review opinion and determining the type of the review opinion, which includes text, voice, image, video, and motion track; After receiving the design review opinions from the review end, the server needs to first check the format of these review opinions and determine their types. Review opinions can be expressed in a variety of ways, including: 1) Text comments: Reviewers can directly enter text comments through the document editor. The server needs to perform natural language processing on the text comments to obtain key information. 2) Voice comments: Reviewers record comments by voice, and the server needs to call the voice recognition module to convert the voice into text information and then process it. 3) Image: Reviewers can take photos or screenshots of the details of the design plan and upload the pictures for annotation. The server needs to analyze the content of the pictures through image recognition technology. 4) Video: Record the operation video of the design plan and upload it to the server. The server can detect video frames and obtain important scenes. 5) Action trajectory: Reviewers make annotations through gestures in the virtual reality environment, and the server obtains gesture trajectory information in real time.

[0068] After receiving the review opinions, the server will check them one by one according to the above types and determine the type of each opinion. An opinion can also contain multiple formats, and the server needs to perform complex analysis. For example, for a voice comment: "The surface processing of this area can be smoother, I think it will be more coordinated", the server needs to call voice recognition first, obtain text information, and then analyze the words "surface" and "smoother" mentioned in the text to determine its semantics, and finally determine that the opinion type is voice type.

[0069] Through format checking, the server can clearly know the specific type of each design review opinion, so as to select the appropriate parsing method for different types in the future to obtain the key content of the review opinion, improve the parsing effect, and provide support for the final generation of the design review report.

[0070] S202, input the design review opinion in a corresponding format according to the review opinion type; After clearly determining the type of design review opinion, the server needs to enter the content of the review opinion according to the corresponding format requirements. Specific entry methods include: For text-type review opinions, the text content of the opinion can be directly extracted and stored in the relevant fields of the review opinion database. For voice types, the voice needs to be transcribed first, the corresponding text manuscript of the voice is obtained, and then the transcribed text content is stored. For image types, the image data object needs to be stored in the binary field of the database, and the image analysis algorithm can be run at the same time to extract the feature vector information of the image, which is convenient for subsequent analysis and retrieval of the image content. For video types, the binary data of the video file also needs to be stored, and video frame extraction can be performed additionally to obtain key frame images and their feature vectors. For motion trajectory types, the point sequence coordinate data of the trajectory needs to be normalized and then stored in a standardized data format.

[0071] When the review comments are entered, the server also needs to store relevant meta-information, such as the provider of the comment and the time of provision, to mark the source and context of each review comment. The summary of the review comments content and meta-information constitutes a complete design review data set.

[0072] Through formatted entry, the server can organize different types of review opinions in a structured form, provide data support for subsequent retrieval, statistics, and analysis, and facilitate the maintenance and management of review opinions to avoid omissions and misunderstandings of information.

[0073] S203, extracting keywords from the received design review opinions; After entering the review opinions, the server needs to extract keywords from the text data (text, speech-to-text) so that the review opinions can be retrieved and analyzed based on keywords later. Specifically, for each text or voice review opinion, the server needs to extract keywords, including Chinese word segmentation, using algorithms such as word sorting and text topic modeling to obtain the keywords of the text. Perform named entity recognition and mark out professional terms such as product components and design features involved in the text as keywords. Use dependency syntactic analysis to extract the evaluation objects and evaluation words of the opinions as keywords. Remove some stop words and single-character words that have no practical meaning. In addition, words with a higher frequency are also retained as keywords.

[0074] After obtaining the keyword set of each review opinion, the server needs to establish an associated index between the review opinion content and the keywords. In the subsequent keyword-based review opinion search, the review opinion that meets the search terms can be quickly located.

[0075] S204, searching the design review opinion database for the input design review opinions containing the keyword; After extracting the keywords of the design review opinions, the server needs to search the review opinion database based on these keywords to find the entered review opinions containing the same or similar keywords.

[0076] Specifically, the server will connect to the review opinion database, which stores all kinds of review opinions entered in all historical reviews and establishes a keyword index. For a newly received review opinion, the server will extract its keywords and construct a structured query language to perform a full-text search in the review opinion database using the keywords. Through this query, all review opinions in the historical review opinion database that contain both the keywords "door curve" and "color needs to be adjusted" can be retrieved. The search results will also include related review opinions with similar keywords, such as opinions that only contain "door curve" or "color needs to be adjusted". These related opinions are also of reference value and can provide reference for the current review.

[0077] The server can also expand the query statement as needed, adding the type of review opinion, product category, etc. as conditions to make the search results more relevant and accurate. By searching historical review opinions based on keywords, similar opinions can be aggregated together to avoid repeated evaluations, and previous relevant experiences can be discovered to improve review efficiency and quality.

[0078] S205, calculating the similarity between the received design review opinion and the retrieved and entered design review opinion; After retrieving relevant historical review opinions through keywords, the server needs to calculate the specific similarity between these existing opinions and the newly received opinions.

[0079] The methods for calculating similarity include: 1) Word vector similarity: Using the word vector algorithm, calculate the cosine similarity of the semantic vectors of the words of the two opinion texts as the similarity score. 2) Keyword coverage: Calculate the percentage of keyword coverage of the text content of the two opinions. The higher the keyword coverage, the closer the opinion topics are. 3) Edit distance: Calculate the minimum number of operations to edit one text into another. The fewer the number of operations, the higher the similarity. 4) Semantic similarity: Use semantic network technology to analyze the semantic content similarity of the two opinion texts.

[0080] The server can combine the above methods to calculate the similarity score between the new and old opinions, that is, the similarity. The similarity can be in the range of 0-1, and the larger the value, the higher the similarity. The similarity calculation can quantitatively determine the degree of duplication of the review opinions, so that the server can automatically identify the historical review opinions that are highly relevant to the new opinions.

[0081] S206: If the similarity is higher than a preset threshold, the received design review opinion is automatically classified into a corresponding input design review opinion item.

[0082] After calculating the similarity between the new review opinion and the historical opinion, the server can automatically classify and process it according to the similarity. Specifically, the server will pre-set a similarity threshold, such as a threshold of 0.8. When it is found that the similarity between a historical review opinion and a new opinion exceeds 0.8, the two opinions are judged to be duplicates. At this time, the server will automatically classify the newly received review opinion into the corresponding historical review opinion item and record it as duplicate content. For example, if the similarity between a new opinion "Door surface treatment affects aerodynamic performance" and a historical opinion is calculated to be 0.85, which exceeds the threshold of 0.8, the server will classify the new opinion under that historical opinion and insert it into its duplicate opinion list.

[0083] If the similarity between the new opinion and all historical opinions is lower than 0.8, a new review opinion record will be created as the new review content. By automatically classifying and processing duplicate content, repeated reviews can be reduced, work efficiency can be improved, and review experts can pay more attention to valuable new opinions. The setting of the similarity threshold needs to be adjusted according to the specific situation. Generally, it can be adjusted in the range of 0.7-0.9 to optimize, which can prevent missed duplications and avoid misjudgment of new opinions.

[0084] In summary, the server can intelligently determine the relevance of new and old review opinions and automatically classify them, making the review opinion library more aggregated and coherent, providing effective support for design review.

[0085] The following describes the pending devices in the embodiments of the present invention from the perspective of hardware processing. Figure 3 , which is a schematic diagram of a physical device structure of the pending equipment in the embodiment of the present application.

[0086] It should be noted that Figure 3 The structure of the undetermined device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0087] like Figure 3As shown, the device to be determined includes a central processing unit (CPU) 301, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 302 or the program loaded from the storage part 308 to the random access memory (RAM) 303, such as executing the method described in the above embodiment. In RAM 303, various programs and data required for system operation are also stored. CPU 301, ROM 302 and RAM 303 are connected to each other through bus 304. Input / output (I / O) interface 305 is also connected to bus 304.

[0088] The following components are connected to the I / O interface 305: an input section 306 including an audio input device, a button switch, etc.; an output section 307 including a liquid crystal display (LCD) and an audio output device, an indicator light, etc.; a storage section 308 including a hard disk, etc.; and a communication section 309 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 309 performs communication processing via a network such as the Internet. A drive 310 is also connected to the I / O interface 305 as needed. A removable medium 311, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 310 as needed so that a computer program read therefrom is installed into the storage section 308 as needed.

[0089] In particular, according to an embodiment of the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present invention includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a computer program for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through the communication part 309, and / or installed from a removable medium 311. When the computer program is executed by the central processing unit (CPU) 301, various functions defined in the present invention are performed.

[0090] It should be noted that specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, apparatus, or device.

[0091] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present invention. Each box in the flowchart or block diagram may represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions marked in the box may also occur in an order different from that marked in the accompanying drawings.

[0092] Specifically, the pending device of this embodiment includes a processor and a memory, and a computer program is stored in the memory. When the computer program is executed by the processor, the AR-based industrial design review method provided in the above embodiment is implemented.

[0093] As another aspect, the present invention further provides a computer-readable storage medium, which may be included in the pending device described in the above embodiment; or may exist independently without being assembled into the pending device. The above storage medium carries one or more computer programs, and when the above one or more computer programs are executed by a processor of the pending device, the pending device implements the AR-based industrial design review method provided in the above embodiment.

[0094] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

[0095] As used in the above embodiments, the term "when..." may be interpreted to mean "if..." or "after..." or "in response to determining..." or "in response to detecting...", depending on the context. Similarly, the phrases "upon determining..." or "if (the stated condition or event) is detected" may be interpreted to mean "if determining..." or "in response to determining..." or "upon detecting (the stated condition or event)" or "in response to detecting (the stated condition or event)", depending on the context.

[0096] Those skilled in the art can understand that to implement all or part of the processes in the above-mentioned embodiments, the processes can be completed by computer programs to instruct related hardware, and the programs can be stored in computer-readable storage media. When the programs are executed, they can include the processes of the above-mentioned method embodiments. The aforementioned storage media include: ROM or random access memory RAM, magnetic disk or optical disk and other media that can store program codes.

Claims

1. An industrial design review method based on AR, applied to a server of a review system, characterized in that: The method comprises: Receive review applications containing three-dimensional industrial designs sent by the user; Extracting data features of the three-dimensional industrial design and matching them with design features stored in a preset database to obtain n similar designs with the highest matching degree, where n is a positive integer set in advance; Based on a preset preliminary design review model, analyzing the differences between the three-dimensional industrial design and the n similar designs, and generating a preliminary review judgment report, wherein the preliminary design review model is a model established by training based on historical design review data; After detecting the review acceptance request sent by the review end, sending the preliminary review judgment report to the review end; Receiving design review opinions sent by the review end in various ways; Analyze the design review opinions through image recognition and natural language processing technology to generate design review results; The design review result is sent to the user end.

2. The method according to claim 1, characterized in that After detecting the review acceptance request sent by the review end, the step of sending the preliminary review judgment report to the review end specifically includes: Receiving a review request including reviewer information sent by the review terminal; After querying the historical review database corresponding to the reviewer information, obtaining the historical review preferences of the reviewer; According to the historical review preferences, corresponding content and precautions are extracted from the preliminary review judgment report to generate personalized review suggestions; The personalized review suggestion is sent to the review end.

3. The method according to claim 1, characterized in that After the step of extracting the data features of the three-dimensional industrial design and matching them with the design features stored in the preset database, the method further includes: The design review opinions, the three-dimensional industrial design and the corresponding product structure data are extracted and marked as different types of entity nodes, and inserted into a preset industrial design knowledge graph; Establishing relationships between entity nodes in the industrial design knowledge graph, and determining the degree of association between corresponding nodes of different three-dimensional industrial designs under the same product category through the reasoning algorithm of the knowledge graph; Determine, according to the correlation, an existing design scheme that has a high correlation with the user's current three-dimensional design scheme; A design optimization scheme is determined according to the existing design scheme, and the design optimization scheme is used to guide the user to improve the current three-dimensional industrial design according to the existing design scheme.

4. The method according to claim 1, characterized in that: After receiving the design review opinions sent by the review end in various ways, the method further includes: Perform a format check on the design review opinion to determine the review opinion type, which includes text, voice, image, video and motion track; and enter the design review opinion in a corresponding format according to the review opinion type.

5. The method according to claim 4, characterized in that After the step of inputting the design review opinion in a corresponding format according to the review opinion type, the method further includes: Extracting keywords of the received design review opinions; retrieving the entered design review opinions containing the keywords in the design review opinion database; The similarity between the received design review opinion and the retrieved entered design review opinion is calculated; if the similarity is higher than a preset threshold, the received design review opinion is automatically classified under the corresponding entered design review opinion item.

6. The method according to claim 1, characterized in that Before the step of receiving the design review opinions sent by the review end in various ways, the step further includes: Receiving reviewer identification information from the review terminal; Determining the review authority of the corresponding reviewer according to the reviewer identification information; If the review authority meets the review requirements, the corresponding review opinions will be received and entered; If the review authority does not meet the review requirements, a message prompting insufficient review authority is sent to the review end.

7. The method according to claim 1, characterized in that The step of receiving the design review opinions sent by the review end in various ways specifically includes: Controlling a virtual reality device to load the virtual model of the three-dimensional industrial design; When receiving a gesture operation performed by a reviewer from the review end, identifying a corresponding gesture trajectory and controlling the virtual model to render the gesture trajectory at a corresponding position; When the reviewer's voice input is detected from the review end, voice recognition is performed to obtain voice text information and the corresponding position of the virtual model is controlled to render the voice annotation; When it is detected that the reviewer from the review end performs an image acquisition operation, the uploaded image is obtained and the image rendering annotation of the corresponding position of the virtual model is controlled; The design review opinions are determined according to the gesture trajectories, the voice annotations, and the picture annotations of each reviewer.

8. A server, characterized in that: The server includes: one or more processors and a memory; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code includes computer instructions, and the one or more processors call the computer instructions to enable the server to execute the method described in any one of claims 1-7.

9. A computer-readable storage medium comprising instructions, characterized in that: When the instructions are executed on a server, the server is caused to execute the method according to any one of claims 1 to 7.

10. A computer program product, characterized in that When the computer program product is run on a server, the server is caused to execute the method according to any one of claims 1 to 7.

Images