Product external inspection ai co-pilot system based on large model, construction method and storage medium

Abstract

The present application belongs to the technical field of appearance detection, and particularly relates to a product appearance inspection AI co-pilot system based on a large model, a construction method and a storage medium. The appearance inspection AI co-pilot system construction method based on a large model comprises the following steps: obtaining enterprise historical experience data, and performing fragmented modular processing on the obtained historical experience data to form an appearance inspection post knowledge base; performing vectorization processing on the data in the appearance inspection post knowledge base through vector model coding, and storing the data in a vector database; establishing a vector index model and a vector recall model; establishing a language interaction relationship between the vector database and the large model through the vector index model and the vector recall model; a user end obtains the content of the vector database through the large model to complete real-time question answering; information is supplemented to the real-time dialogue data of the user end, and dialogue data is supplemented with key information extraction to the vector database. The present application provides standardized work guidance for on-site appearance inspection posts to improve appearance inspection defect recognition efficiency and disposal efficiency.

Description

Technical Field

[0001] This invention belongs to the field of appearance inspection technology, specifically relating to an AI co-pilot system for product appearance inspection based on a large model. Background Technology

[0002] In the production and maintenance of steel products, quality and appearance inspection are required. Currently, the appearance inspection process for steel products requires manual inspection of each finished product in the warehouse. The inspection relies on manual experience to discover and judge appearance defects. Common defect types include cracks, dents, scratches, etc., and each major defect category has several subcategories.

[0003] Traditional product appearance inspection methods rely on manual labor and have the following drawbacks:

[0004] (1) The methods and results of identifying defects in appearance inspection vary from person to person and lack unified standard guidance;

[0005] (2) Experienced operators are required to accurately determine the type of defect, its cause, and the appropriate handling measures;

[0006] (3) For situations that are difficult to identify and judge, it is necessary to organize experienced technical personnel to make a comprehensive judgment, which is inefficient;

[0007] (4) The experience in appearance inspection is limited to the minds of technical personnel or computers and has not yet been formed into a corporate knowledge base;

[0008] (5) Relying on the traditional master-apprentice method, the training cycle for newcomers is long.

[0009] Traditional product appearance inspection methods make it difficult for the extensive experience of many outstanding individuals to be fully shared and passed on. While existing defect detection methods rely on cameras and deep learning modules for automatic detection, they suffer from inaccurate recognition and judgment, leading to inaccurate detection results.

[0010] Based on the above problems, the applicant has developed an AI-assisted external inspection system for the steel industry that can transform historical experience in appearance inspection into experiential knowledge and form a unified standard for appearance inspection. This system is applied to the standardized operation guidance of on-site appearance inspection positions to improve the efficiency of appearance inspection defect identification and handling. Summary of the Invention

[0011] The purpose of this invention is to overcome the shortcomings of existing technologies, such as the lack of standardized experience in appearance inspection and the varying efficiency and accuracy of inspections by individuals. It provides a product appearance inspection AI co-pilot system based on a large model, a construction method, and a storage medium for standardized operation guidance for on-site appearance inspection positions.

[0012] The technical solution adopted by this invention to solve its technical problem is:

[0013] The first aspect is a method for constructing an external inspection AI co-pilot system based on a large model, which includes the following:

[0014] Acquire historical experience data from enterprises, and process the acquired historical experience data into segments and modules to form a knowledge base for external inspection positions;

[0015] The data in the knowledge base for external inspection positions is vectorized using vector model encoding and stored in a vector database;

[0016] Establish a vector indexing model and a vector recall model, that is, to label and edit the data in the vector database, and use the data stored in the vector database to label and recall questions and answers;

[0017] The vector database and the large model establish a language interaction relationship through the vector indexing model and the vector recall model; the user asks questions through the large model to obtain the content of the vector database and completes real-time question answering.

[0018] Information is supplemented to the real-time dialogue data from the user's end, and key information is extracted from the dialogue data and added to the vector database.

[0019] Specifically, the acquired historical experience data includes one or more of the following: image data, text data, voice data, video data, and report documents.

[0020] The acquired historical experience data is processed in a segmented and modular manner, including the following:

[0021] Separate historical experience data according to data type;

[0022] The separated image data and / or text data and / or audio data and / or video data are organized and cleaned, and processed into a unified standard format;

[0023] The data in the standard format is encoded and segmented according to its length, and then divided into several smaller blocks according to semantics.

[0024] Specifically, the data in the external inspection post knowledge base undergoes vectorization processing via vector model encoding, including the following:

[0025] Natural language processing technology is used to parse the text information that is semantically divided into several small blocks, and vector units for natural language understanding and reasoning are established through a model encoder;

[0026] Computer vision algorithms are used to identify and label defects in the image information that is semantically segmented into several small blocks, and the generated label information is converted into vector form.

[0027] Specifically, the defect annotation includes either a defect frame annotation or a defect arrow annotation.

[0028] Specifically, the process of supplementing the real-time dialogue data from the user's end with information and extracting key information from the dialogue data to supplement the vector database includes the following:

[0029] Dynamically adjust the size of the client context window;

[0030] Information extraction technology is used to extract key information from each round of dialogue and process it into vectors;

[0031] The key information after vectorization is supplemented and stored in the vector database.

[0032] Specifically, the dialogue data includes one or more of the following: defect type data, cause data, solution data, and example images.

[0033] The second aspect is a product external inspection AI co-pilot system based on a large model, including:

[0034] Enterprise historical experience data files; the historical experience data files include at least one of image data, text data, voice data, and video data;

[0035] The document parsing and splitting module parses the historical experience data file and splits it into image data and / or text data and / or audio data and / or video data;

[0036] The data vectorization module is used to vectorize the data after it has been split and categorized.

[0037] A vector database is used to store the data processed by the data vectorization module.

[0038] The AI ​​large model serves as the client input port and matches the data information of the index question with the vector database through the vector indexing module, and queries the data information of the question through the vector recall module;

[0039] The information supplementation module includes a dynamic adjustment unit and a key information extraction unit; the dynamic adjustment unit is used to dynamically adjust the size of the user-side context window, and the key information extraction unit is used to extract key information from the dialogue.

[0040] Specifically, it also includes a quantization annotation module and a retrieval enhancement module, wherein the vectorization annotation module is used to provide quantization annotations to the administrator backend and / or users.

[0041] Specifically, it also includes a data detection and analysis module, which is used to analyze the usage data and calculate and display key indicators; the key indicators include at least one of the following: number of users, number of active users, large language model call consumption, usage type trend, total number of messages, average number of appropriate interactions, and output speed.

[0042] Thirdly, a storage medium storing processor-executable instructions, which, when executed by a processor, are used to perform a method for constructing a product external inspection AI co-pilot system based on a large model, as described above.

[0043] The beneficial effects of the AI ​​co-pilot system for product external inspection based on a large model, its construction method, and its storage medium of the present invention are as follows:

[0044] This invention segments and modularizes text and image data from historical experience data. Specifically, it processes text and image data independently by adapting their respective features to form an external inspection job knowledge base. This improves the accuracy of the data within the knowledge base. The data in the external inspection job knowledge base is then vectorized and stored. Semantic vectorization technology is used to improve retrieval efficiency and accuracy, making content-based retrieval more precise and effective. Furthermore, by establishing a vector indexing module and a vector recall module, language interaction between the vector database and the large model is achieved, providing standardized operational guidance for on-site appearance inspection positions to improve the efficiency of appearance inspection defect identification and handling.

[0045] This invention supplements user-side dialogue data with information, preventing the unusability of previous dialogue content due to dialogue exceeding contextual limitations. By dynamically adjusting the context window, it flexibly adapts to different lengths of dialogue history according to actual dialogue needs, ensuring no information loss and guaranteeing information integrity and coherence. Furthermore, this invention extracts key information from user-side dialogue data and supplements it into vector database 4, ensuring real-time updates to the data in vector database 4, improving the accuracy and relevance of responses. In subsequent dialogues, this vectorized key information can be queried to maintain dialogue coherence and ensure dialogue quality, thereby enhancing user experience and improving system flexibility and responsiveness. Attached Figure Description

[0046] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0047] Figure 1 This is a flowchart of the construction method of the product external inspection AI co-pilot system in this embodiment.

[0048] Figure 2 This is a diagram illustrating the construction process of the AI ​​co-pilot system for product external inspection in this embodiment.

[0049] Figure 3 This is a simplified diagram of the formation of the vector database in this embodiment.

[0050] Figure 4 This is a diagram of the application framework of the large model in this embodiment.

[0051] Figure 5 This is a flowchart of the segmented and modular processing of historical experience data in this embodiment.

[0052] Figure 6 This is a flowchart of the information supplement section in this embodiment.

[0053] Figure 7 This is a screenshot of the software interface for natural language dialogue in this embodiment.

[0054] Figure 8 This is a screenshot of the software interface for responding to the defect labeling of crack defects in Embodiment 1.

[0055] Figure 9 This is a schematic diagram of the product external inspection AI co-pilot system in Embodiment 2.

[0056] Figure 10 This is a diagram showing the data analyzed by the data detection and analysis module in this embodiment two on the software interface.

[0057] The diagram shows: 1. Enterprise historical experience data file, 2. Document parsing and splitting module, 3. Data vectorization module, 4. Vector database, 5. AI large model, 6. Vector indexing module, 7. Vector recall module, 8. Information supplementation module, 81. Dynamic adjustment unit, 82. Key information extraction unit, 9. Quantitative annotation module, 10. Search enhancement module, and 11. Data detection and analysis module. Detailed Implementation

[0058] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0059] Example 1

[0060] This embodiment takes the appearance inspection of products in the steel industry as an example. Figures 1-8 The specific embodiment of the construction method of the product external inspection AI co-pilot system based on a large model of the present invention shown includes the following:

[0061] S10: Acquire historical experience data from enterprises, and process the acquired historical experience data into segments and modules to form a knowledge base for external inspection positions in the steel industry;

[0062] S20: Data in the knowledge base of external inspection positions is vectorized through vector model encoding and stored in vector database 4;

[0063] S30: Establish a vector indexing model and a vector recall model, that is, to label and edit the data in vector database 4, and use the data stored in vector database 4 to label and recall questions and answers;

[0064] S40: Establish a language interaction relationship between the vector database and the large model through the vector indexing model and the vector recall model; the user terminal asks questions through the large model to obtain the content of the vector database and completes real-time question answering.

[0065] S50: Supplement the real-time dialogue data from the user end with information and extract key information from the dialogue data to supplement the vector database 4.

[0066] This embodiment segments and modularizes the text and image data from historical experience data. Specifically, it processes the text and image data independently by adapting them to their respective features, forming an external inspection job knowledge base. This improves the accuracy of the data within the external inspection job knowledge base. The data in the external inspection job knowledge base is then vectorized and stored. Semantic vectorization technology is used to improve retrieval efficiency and accuracy, making content-based retrieval more precise and effective. Furthermore, by establishing a vector index module 6 and a vector recall module 7, language interaction between the vector database 4 and the large model is achieved, providing standardized operational guidance for on-site appearance inspection positions to improve the efficiency of appearance inspection defect identification and handling.

[0067] The historical experience data acquired in step S10 includes one or more of image data, text data, voice data, and video data. In this embodiment, the enterprise's historical experience data includes image data and text data related to the appearance inspection of the steel industry. It should be understood that the historical experience data in this embodiment only lists text data and image data, but is not limited to text data and image data. Among them, appearance defects in the steel industry include cracks, dents, scratches, etc. Cracks can be further refined according to the causes of crack formation. Dents and scratches are further subdivided into smaller categories based on their different sizes and dimensions to ensure the completeness and accuracy of the data. This embodiment uses the appearance of steel industry products as an example, but it can also be applied to other industries, and no absolute limitation is made here.

[0068] To further clarify, see [link / reference] Figure 5 In this embodiment, the historical experience data is processed in a segmented and modular manner, including the following:

[0069] S101: Separate historical experience data according to data type;

[0070] S102: Organize and clean the separated image data and / or text data and / or voice data and / or video data, and process them into a unified standard format;

[0071] S103: Encode the data after standard formatting and divide the data into segments according to the data length, and divide it into several small blocks according to semantics.

[0072] Step S10 extracts the company's accumulated historical experience to form a high-quality corpus for use as input to the large model, including text and image data. This corpus is then segmented and modularized to construct an external inspection job knowledge base. This knowledge base integrates the work experience of multiple experienced technical personnel, forming an enterprise-level external inspection knowledge base. Its content can be flexibly maintained and expanded as experience accumulates. This knowledge base is primarily used for on-site operational guidance and new employee training, greatly facilitating knowledge sharing and learning among internal personnel, and contributing to knowledge sharing and experience transfer.

[0073] In step S20, the data in the knowledge base of the external inspection post is vectorized through vector model encoding. Natural language processing technology is used to parse the text information that is divided into several small blocks according to semantics. Vector units for natural language understanding and reasoning are established through model encoder. Computer vision algorithms are used to identify and label defects in the image information that is divided into several small blocks according to semantics, and the generated label information is converted into vector form.

[0074] In this embodiment, text and image data in the external inspection post knowledge base are processed separately. Vectorization is performed using a suitable vector model encoding and module. Natural language processing (NLP) technology is used to parse the text data; specifically, a model encoder is used to establish vector units for natural language understanding and reasoning. Computer vision algorithms are employed to identify and label defects in the image data, and the generated label information is converted into vector form. The vectorized text and image data are stored in vector database 4. Semantic vectorization technology is used to improve retrieval efficiency and accuracy, making content-based retrieval more precise and effective. This meticulous multimodal information processing strategy not only enhances the response speed of retrieval operations but also improves the relevance and accuracy of retrieval results, thereby optimizing the overall information management process.

[0075] Defect annotation includes either defect frame annotation or defect arrow annotation. In this embodiment, defect annotation uses defect frame annotation, such as... Figure 8 As shown, this is a defect label with a crack.

[0076] The data stored in Vector Database 4 is used to annotate and recall questions and answers, increasing the accuracy of the answers. Humans edit and correct the annotated and recalled data in the background to ensure the accuracy of the dialogue data.

[0077] In step S40 of this embodiment, a vector indexing model and a vector recall model are set up between the vector database 4 and the large model to establish a language interaction relationship between the vector database 4 and the large model. The constructed vector indexing model is used to match and index the question and answer data. The vector recall model is used to recall the dialogue data, edit and correct it, and also to obtain the job guidance information in real time and update the data stored in the vector database 4.

[0078] This embodiment uses a large model as the user end, leveraging its strong semantic understanding and correlation analysis capabilities to achieve efficient interactive question-and-answer in the form of natural language or voice-to-speech. Step S50 involves supplementing the real-time dialogue data from the user end with information and extracting key information from the dialogue data to supplement the vector database 4, such as... Figure 6 As shown, the specific content is as follows:

[0079] S501: Dynamically adjust the size of the client context window;

[0080] S502: Employ information extraction technology to extract key information from each round of dialogue and process it into vectors;

[0081] S503: The key information after vectorization is supplemented and stored in the vector database 4.

[0082] When handling multi-turn dialogues, if the dialogue exceeds the model's contextual limitations, previous dialogue content cannot be effectively utilized, impacting user experience. To address this issue, we can adopt two optimization approaches: First, by dynamically adjusting the size of the model's context window, the model can flexibly adapt to different lengths of dialogue history based on actual dialogue needs, ensuring no relevant information is lost. Second, information extraction techniques can be used to extract and vectorize key information from each turn of dialogue. This way, in subsequent dialogues, even if the original contextual limitations are exceeded, the system can maintain dialogue coherence by querying this vectorized key information, improving the accuracy and relevance of responses, thereby enhancing user experience. Combining these two methods can enhance the system's flexibility and responsiveness while maintaining dialogue quality.

[0083] This embodiment leverages the semantic understanding and association analysis capabilities of a large-scale model. By dynamically adjusting the size of the user-side context window—that is, through multi-turn dialogue context management optimization—it achieves efficient interactive question-and-answer based on natural language or voice intercom. It can also obtain standardized operational guidance information related to product surface inspection in real time, such as... Figure 7 and Figure 8 As shown, the documents include representative images of the relevant defects, their causes and remedial measures, as well as analysis reports of similar historical defects, which promotes more efficient external inspection operations.

[0084] This embodiment summarizes and refines historical data scattered in human brains or computers, and through large-scale model semantic understanding and vectorization processing, establishes a multimodal document parsing system in the preprocessing stage to form an external inspection knowledge base, including text, images, etc., to support standardized operation guidance. By constructing a unified knowledge base management system, it realizes the maintenance and management of relevant texts and images, including debugging question and answer results, adding annotations, etc., to achieve knowledge management such as information updates and authorized objects, as well as experience transfer.

[0085] Example 2

[0086] like Figure 9 and Figure 10 As shown, based on the above-mentioned construction method of the external inspection AI co-pilot system, the external inspection AI co-pilot system based on the same inventive concept includes:

[0087] Enterprise historical experience data file 1; the historical experience data file includes at least one of the following: image data, text data, audio data, video data, and report documents;

[0088] Document parsing and splitting module 2 parses historical experience data files and splits them into image data and / or text data and / or audio data and / or video data and / or report documents;

[0089] Data vectorization module 3 is used to vectorize the data after it has been split and classified.

[0090] Vector database 4 is used to store the data processed by data vectorization module 3;

[0091] AI large model 5 serves as the client input port and, together with vector database 4, uses vector indexing module 6 and vector recall module 7 to index and match the data information of the query question.

[0092] The information supplementation module 8 includes a dynamic adjustment unit 81 and a key information extraction unit 82. The dynamic adjustment unit 81 is used to dynamically adjust the size of the user terminal context window, and the key information extraction unit 82 is used to extract key information from the dialogue.

[0093] This embodiment uses historical experience data file 1, which includes text data, image data, and report documents, as an example. In this embodiment, the AI ​​large model 5 adopts the qwen2.572B large model (Transformer architecture) as the core engine of the entire system. Users can directly and quickly obtain relevant text and image information from the knowledge base through natural language dialogue or voice intercom, achieving standardized job operation guidance. Serving as a readily available AI co-pilot to assist in efficient operations, the data is converted into vector patterns and stored in the vector database 4. The vector indexing module 6 and vector recall module 7 are used to index and match the data information for querying questions, achieving language interaction and real-time question answering. The actual question answering effect is as follows: Figure 7 and Figure 8 As shown, this system can obtain standardized operating instructions related to product surface inspection in real time, including representative pictures of relevant defects, causes and handling measures, as well as text information such as analysis reports of similar historical defects.

[0094] Information Supplement Module 8 creates system applications for different domains from data documents categorized by type, for use by specialized personnel. Document content is supplemented, uploaded, and then encoded again before being stored in the database for data updates and processing, facilitating retrieval results. The Information Supplement Module 8 allows users to query and edit both user-asked questions and answers, saving the modified answers to Vector Database 4. This serves two purposes: firstly, it creates data storage space for different domains, facilitating user queries; secondly, it updates the data in Vector Database 4 in real time, ensuring accurate answers for user searches and for the large model's summary.

[0095] This system also includes a quantitative annotation module 9 and a search enhancement module. The quantitative annotation module 9 is used to provide quantitative annotations to the administrator backend and / or users. The search enhancement module is used to filter and bind the correct answers based on the quantitative annotations of the administrator backend standards and / or user likes and dislikes, thereby enhancing the index matching accuracy.

[0096] The Quantization Annotation Module 9 enables the system to provide annotated response functionality, offering customized, high-quality question-and-answer capabilities for applications through manual editing. It can directly provide predefined answers without the need for large language models (LLM) generation, improving the accuracy and reliability of responses and effectively avoiding content illusion problems that may occur during retrieval-based generation (RAG). Specifically, the annotated response function correctly binds and stores questions and answers in a vector library, increasing retrieval accuracy. Quantization annotation through administrator backend standards or user likes / dislikes further enhances search accuracy, providing customized, high-quality question-and-answer capabilities for applications. The steps to avoid content illusion problems are as follows:

[0097] A10: Standardize and organize the formats of text and image data, and add special marks and symbols;

[0098] A20: Performs intelligent slicing of text data by length, adjusting the length to optimize storage;

[0099] A30: Vector storage for retrieval filtering and binding of correct answer annotations to exclude illusion problems.

[0100] By adding a quantitative annotation module (Module 9) to the image data, the system's stability and reliability are enhanced, ensuring users receive more accurate and consistent information services. Figure 8 The image data shown is labeled.

[0101] The retrieval enhancement module sets up a closed loop for question-and-answer retrieval, answer generation, and feedback. Specifically, it includes the following processing steps: data organization and cleaning, and a structured, standardized format; intelligent slicing that divides long documents into semantic blocks according to length to improve retrieval granularity; and storing data vectors, combined with word segmentation indexes, in a vector database.

[0102] To better present the usage and key data of this system, this system also includes a data detection and analysis module, which is used to analyze the usage data and calculate and display key indicators; key indicators include at least one of the following: number of users, number of active users, large language model call consumption, usage type trend, total number of messages, average number of appropriate interactions, and output speed.

[0103] The data monitoring and analysis module displays key metrics such as user volume, active user count, and Large Language Model (LLM) call consumption. This data provides a solid foundation for continuous application optimization, user activity, and cost-effectiveness. By monitoring user volume and active user count, we can gain deeper insights into application usage trends and user behavior patterns, thereby developing more precise operational strategies to improve user engagement. Simultaneously, analyzing the call consumption of the AI ​​Large Language Model 5 helps assess cost-effectiveness, optimize resource allocation, and improve service economics. These insights collectively promote continuous application optimization, ensuring more efficient operation and management while meeting user needs. See [link to relevant documentation]. Figure 10 This is a data chart for actual use.

[0104] This invention segments and modularizes the text and image data in the enterprise historical experience data file 1 using the document parsing and splitting module 2 to form an external inspection post knowledge base. The data in the external inspection post knowledge base is then vectorized and stored in the vector database 4 using the data vectorization module 3. Semantic vectorization technology is used to improve retrieval efficiency and accuracy, making content-based retrieval more precise and effective. Furthermore, the vector database 4 and the AI ​​large model 5 are enabled to interact through the establishment of the vector index module 6 and the vector recall module 7, providing standardized operation guidance for on-site appearance inspection posts.

[0105] Meanwhile, the quantitative annotation module 9 enhances the stability and reliability of the system by annotating and responding to image data. The retrieval enhancement module creates a closed loop between question-and-answer retrieval, answer generation, and feedback, improving retrieval accuracy and thus increasing the efficiency of defect identification and handling in appearance inspection. Furthermore, the data detection and analysis module monitors key indicators such as user volume, active user count, and Large Language Model (LLM) call consumption, ensuring the system achieves more efficient operation and management while meeting user needs. It should be understood that this system can not only be directly applied in external inspection but also provide training and development opportunities for new employees, featuring rich graphics and natural language interaction for a more intuitive and efficient experience.

[0106] Example 3

[0107] The construction method described in Example 1 and the system described in Example 2 are written into corresponding computer code and stored in a storage medium. When the storage medium is connected to the controller, the computer program code therein can be read out and executed, thereby automatically executing steps S10-S50 and achieving the same technical effect as in Example 1 or Example 2.

[0108] It should be recognized that embodiments of the present invention can be implemented by computer hardware, a combination of hardware and software, or by computer instructions stored in a non-transitory computer-readable storage medium, or the implementation methods can be implemented using standard programming techniques—including implementation in a computer program on a non-transitory computer-readable storage medium configured with a computer program, wherein such a storage medium causes the computer to operate in a particular and predefined manner—according to the methods and drawings described in the specific embodiments. Each program can be implemented in a high-level procedural or target-terminal programming language to communicate with a computer system. However, if desired, the program can be implemented in assembly or machine language. In any case, the language can be a compiled or interpreted language. Furthermore, for this purpose, the program can run on a programmed application-specific integrated circuit.

[0109] It should be understood that the specific embodiments described above are for illustrative purposes only and are not intended to limit the scope of the invention. Obvious variations or modifications derived from the spirit of the invention are still within the protection scope of the invention.

Claims

1. A method for constructing an external inspection AI co-pilot system based on a large model, characterized in that, The application comprises the following contents: Acquiring enterprise historical experience data and performing fragmented modular processing on the acquired historical experience data to form an external inspection post knowledge base; Vectorizing the data in the external inspection post knowledge base through vector model coding and storing the data in a vector database; Establishing a vector index model and a vector recall model, i.e., annotating and editing the data in the vector database, and annotating and recalling problems and answers using the data stored in the vector database; Establishing a language interaction relationship between the vector database and a large model through the vector index model and the vector recall model; a user end obtains the content of the vector database through a large model to complete real-time question answering; Supplementing information to real-time conversation data of the user end and extracting key information from the conversation data and supplementing the key information to the vector database; The acquired historical experience data comprises one or more of picture data, text data, voice data and video data; The fragmented modular processing of the acquired historical experience data comprises the following contents: Separating the historical experience data according to data types; Organizing and cleaning the separated picture data and / or text data and / or voice data and / or video data and processing them into a unified standard format; Encoding the data in the standard format and separating and fragmenting the data length, and separating them into several small blocks according to semantics.

2. The method according to claim 1, wherein The vectorization processing of the data in the external inspection post knowledge base through the vector model coding comprises the following contents: Analyzing the text information in the several small blocks separated according to semantics by using natural language processing technology, and establishing a natural language understanding and reasoning vector unit through a model encoder; Using a computer vision algorithm to identify and label defects in the picture information in the several small blocks separated according to semantics, and converting the generated label information into a vector form.

3. The method according to claim 2, wherein: The defect labeling comprises any one of defect bounding box labeling and defect arrow labeling.

4. The method according to claim 1, wherein the method is characterized by: The supplementing of information to real-time conversation data of the user end and the extraction of key information from the conversation data and the supplementing of the key information to the vector database comprise the following contents: Dynamically adjusting the size of the context window of the user end; Using information extraction technology to extract key information in each round of conversation and perform vectorization processing on the key information; Supplementing and storing the vectorized key information in the vector database. The conversation data comprises one or more of defect type data, cause data, solution data and example pictures.

5. The method according to claim 4, wherein: The application comprises the following contents:

6. A large model-based product external inspection AI co-pilot system, characterized by, An enterprise historical experience data file; the historical experience data file comprises at least one of picture data, text data, voice data and video data; A document parsing and splitting module, which parses a document and splits out picture data or / and text data or / and voice data or / and video data or / and a report document from the historical experience data file; A data vectorization module, which is used to perform vectorization processing on the classified and split data; A vector database, which is used to store the data processed by the data vectorization module; ​ The AI large model is a client input port, and matches index data information of the question through a vector index module and queries data information of the question through a vector recall module. The information supplementing module comprises a dynamic adjusting unit and a key information extracting unit; the dynamic adjusting unit is used for dynamically adjusting the size of the user end context window, and the key information extracting unit is used for extracting key information in the dialogue.

7. The large model-based product external inspection AI co-pilot system according to claim 6, characterized in that: Further comprising a vectorization labeling module and a retrieval enhancement module, the vectorization labeling module is used for providing quantitative labeling for the administrator background or / and the user.

8. The product external inspection AI co-pilot system based on a large model according to claim 6, characterized in that: Further comprising a data detection and analysis module, which is used for analyzing the use data and calculating key indicators for display; the key indicators include at least one of user quantity, active user number, large language model calling consumption, use type trend, total message number, average appropriate interaction number and output speed.

9. A storage medium having stored therein processor-executable instructions, which, when executed by a processor, cause the processor to perform: The instructions executable by the processor, when executed by the processor, are used to perform a construction method of a large model-based product external inspection AI copilot system according to any one of claims 1-5.

Images