Product production method, device and equipment based on large model and storage medium

Abstract

The invention discloses a product production method and device based on a large model, equipment and a storage medium, and relates to the technical field of artificial intelligence, and the method comprises the steps: collecting initial multi-modal data corresponding to a target production task, and obtaining target multi-modal data based on the initial multi-modal data; using different neural networks to extract feature information of different modal production data in the target multi-modal data, and fusing different feature information to obtain target multi-modal features; and generating a real-time production analysis result based on the target multi-modal features, the target multi-modal large model and a large model quantification technology, obtaining different types of target decisions corresponding to the target production task according to the production analysis result, and carrying out product production according to the target decisions. According to the method, the multi-modal production data is collected and fused, so that the production activity is guided by utilizing the multi-modal data, and the problem of low production efficiency caused by the fact that only single-modal data is analyzed for production is avoided.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a product production method, device, equipment and storage medium based on a large model. Background Art

[0002] The core of intelligent manufacturing is to use the deep integration of information technology and physical systems to achieve a high degree of automation and intelligence in the production process. In this process, a large amount of multimodal data is generated, including but not limited to images, text, sound, sensor data, etc. The efficient processing and utilization of this data is crucial to improving production efficiency and ensuring product quality.

[0003] At present, in the field of intelligent manufacturing, single-modal data is usually used to guide production activities. This method cannot integrate multimodal data, and thus cannot fully utilize the complementary advantages of multimodal data to guide production activities, resulting in unreasonable production methods and reduced production efficiency. Summary of the invention

[0004] In view of this, the purpose of the present invention is to provide a product production method, device, equipment and storage medium based on a large model, which can solve the problem of not being able to use the complementary advantages of multimodal data to guide production due to processing only single modal data by fusing multimodal production data. The specific scheme is as follows:

[0005] In a first aspect, the present application provides a product production method based on a large model, comprising:

[0006] Collecting initial multimodal data corresponding to a target production task from a target production line, and acquiring target multimodal data corresponding to the target production task based on the initial multimodal data;

[0007] Using different neural networks to extract different feature information corresponding to production data of different modes in the target multimodal data, and fusing the different feature information to obtain target multimodal features;

[0008] Based on the target multimodal features, the target multimodal large model and the preset large model quantification technology, a real-time production analysis result corresponding to the target production task is generated, different types of target decisions corresponding to the target production task are obtained according to the production analysis result, and product production is performed according to the target decision.

[0009] Optionally, the acquiring target multimodal data corresponding to the target production task based on the initial multimodal data includes:

[0010] A data cleaning operation is performed on the initial multimodal data to obtain corresponding cleaned multimodal data, and a data format conversion is performed on the cleaned multimodal data to obtain the target multimodal data corresponding to the initial multimodal data.

[0011] Optionally, fusing the different feature information to obtain the target multimodal feature includes:

[0012] Dynamically adjust the weights of different feature information according to the actual application scenario of the target multimodal large model and a preset weight adjustment algorithm;

[0013] Based on the weights of the different feature information and the attention mechanism, the different feature information are weightedly fused to obtain the target multimodal features.

[0014] Optionally, the large model-based product production method further includes:

[0015] The target multimodal large model is deployed using a cloud-edge collaborative deployment method, and the target multimodal large model is continuously monitored using a preset monitoring tool to regularly retrain the target multimodal large model.

[0016] Optionally, the large model-based product production method further includes:

[0017] An online learning capability is configured for the target multimodal large model so that the target multimodal large model automatically adjusts model parameters according to real-time feedback information of the production environment corresponding to the target production task.

[0018] Optionally, the large model-based product production method further includes:

[0019] Collecting historical initial multimodal data corresponding to different historical production tasks from the target production line, and performing data cleaning operations on the historical initial multimodal data to obtain corresponding cleaned historical multimodal data;

[0020] Annotating the cleaned historical multimodal data according to the data type of each modal data in the cleaned historical multimodal data, and enhancing the annotated historical multimodal data according to the data type of each modal data to obtain historical target multimodal data;

[0021] A training set is constructed according to the historical target multimodal data, and an initial multimodal large model is trained using the training set to obtain the target multimodal large model.

[0022] Optionally, after the initial multimodal large model is trained using the training set, the method further includes:

[0023] A test set corresponding to the target multimodal large model is constructed according to the historical target multimodal data corresponding to the historical production tasks, and the trained target multimodal large model is evaluated using the test set.

[0024] In a second aspect, the present application provides a product production device based on a large model, comprising:

[0025] A data acquisition module, used to collect initial multimodal data corresponding to a target production task from a target production line, and acquire target multimodal data corresponding to the target production task based on the initial multimodal data;

[0026] An information fusion module, used to extract different feature information corresponding to production data of different modes in the target multimodal data by using different neural networks, and fuse the different feature information to obtain target multimodal features;

[0027] A decision acquisition module is used to generate real-time production analysis results corresponding to the target production task based on the target multimodal features, the target multimodal large model and the preset large model quantization technology, obtain different types of target decisions corresponding to the target production task according to the production analysis results, and produce products according to the target decisions.

[0028] In a third aspect, the present application provides an electronic device, including:

[0029] Memory, used to store computer programs;

[0030] A processor is used to execute the computer program to implement the aforementioned large model-based product production method.

[0031] In a fourth aspect, the present application provides a computer-readable storage medium for storing a computer program, which, when executed by a processor, implements the aforementioned large model-based product production method.

[0032] In this application, firstly, the initial multimodal data corresponding to the target production task is collected from the target production line, and the target multimodal data corresponding to the target production task is obtained based on the initial multimodal data, and then different neural networks are used to extract different feature information corresponding to the production data of different modes in the target multimodal data, and the different feature information is fused to obtain the target multimodal features, and finally, based on the target multimodal features, the target multimodal large model and the preset large model quantification technology, the real-time production analysis results corresponding to the target production task are generated, and different types of target decisions corresponding to the target production task are obtained according to the production analysis results, and the product is produced according to the target decision. It can be seen that this application solves the problem of not being able to use the complementary advantages of multimodal data to guide production due to processing only single modal data by collecting multimodal production data and fusing the feature information corresponding to the multimodal production data; by generating the target decision corresponding to the target production task according to the multimodal features, and using the target decision to guide the production task, the rationality of the production process is guaranteed, thereby improving production efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0034] Figure 1 A flow chart of a product production method based on a large model disclosed in this application;

[0035] Figure 2 A schematic diagram of a multi-modal large model structure disclosed in this application;

[0036] Figure 3 A framework diagram of a multi-modal large model system disclosed in this application;

[0037] Figure 4 A flow chart of a multimodal large model training method disclosed in this application;

[0038] Figure 5 This is a schematic diagram of the structure of a product production device based on a large model disclosed in this application;

[0039] Figure 6 This is a structural diagram of an electronic device disclosed in this application. DETAILED DESCRIPTION

[0040] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0041] At present, the methods for guiding production activities in the field of intelligent manufacturing have the problem of being unable to fuse multimodal data and unable to fully utilize the complementary advantages of multimodal data to guide production activities. To this end, the present application provides a product production method based on a large model, which solves the problem of being unable to utilize the complementary advantages of multimodal data to guide production by fusing the feature information corresponding to multimodal production data.

[0042] See also Figure 1 As shown, the embodiment of the present invention discloses a product production method based on a large model, comprising:

[0043] Step S11: collecting initial multimodal data corresponding to a target production task from a target production line, and acquiring target multimodal data corresponding to the target production task based on the initial multimodal data.

[0044] In this embodiment, the target multimodal large model includes: Figure 2 The data preprocessing module, feature extraction module, multimodal fusion module and decision support module shown in FIG. 1 are responsible for different data processing functions. The framework of the target multimodal large model is shown in FIG. Figure 3 As shown, it includes functions such as data collection and preprocessing, feature extraction, and multimodal fusion. Among them, the data collection and preprocessing function is responsible for collecting multimodal data, namely, initial multimodal data, from multimodal data sources such as various sensors, visual devices, audio recorders, production logs, and equipment status monitoring systems on the production line, including but not limited to images, videos, audio, text, sensor data, etc., to ensure that the data coverage is comprehensive and covers all stages and links in the production process; after collecting the initial multimodal data, the initial multimodal data needs to be processed.

[0045] In this embodiment, the process of acquiring target multimodal data based on initial multimodal data specifically includes: performing data cleaning operations on the initial multimodal data to obtain corresponding cleaned multimodal data, and performing data format conversion on the cleaned multimodal data to obtain target multimodal data corresponding to the initial multimodal data; specifically, the collected original data, i.e., the initial multimodal data, is cleaned, format converted and standardized to ensure data quality, eliminate noise, and convert data of different modalities into a format that can be input into a deep learning model. For example, the cleaned multimodal data can be converted into a numerical vector to enhance the understanding and learning ability of the model; data cleaning The process is to check and remove outliers and noise in the data; handle missing values ​​and fill them with interpolation or other methods; standardize or normalize numerical data to ensure consistency between different data sources; and the above data processing operations, in addition to data cleaning and format conversion, also include: data annotation, that is, necessary annotation work for image, video and audio data, such as target detection boxes, category labels, etc.; semantic annotation or sentiment analysis for text data; data enhancement, that is, rotating, scaling, flipping and other operations on image data to increase data diversity; adding noise and speed changes to audio data; synonym replacement and sentence reorganization for text data. By performing data cleaning, data annotation and data format conversion operations on the collected initial multimodal data, the reliability of the obtained target multimodal data is ensured, so that the target multimodal data can be better understood by the large model.

[0046] Step S12: different neural networks are used to extract different feature information corresponding to production data of different modes in the target multimodal data, and different feature information is fused to obtain target multimodal features.

[0047] In this embodiment, first, different neural networks need to be used to extract feature information corresponding to data of different modalities, that is, deep neural networks for each modality characteristic (such as CNN (Convolutional Neural Networks, a convolutional neural network) for images, RNN (Recurrent Neural Network) or LSTM (Long Short-Term Memory, i.e., long short-term memory network) for time series data) are used to extract feature information; specifically, for image data, pre-trained convolutional neural networks (such as ResNet, Inception, etc.) are used to extract image features; a transfer learning strategy is adopted to fine-tune the network structure according to the actual application scenario; for text data, pre-trained natural language processing models (such as BERT (Bidirectional Encoder Representations from Transformers, a natural language processing model), RoBERTa, etc.) are used to extract text features, and the model size and number of layers can be adjusted as needed during feature extraction to adapt to different computing resource constraints; for signal data, RNN, LSTM or GRU (Gateway Representations from Transformers, a natural language processing model) are used. RecurrentUnit, gated recurrent unit) processes time series data; applies wavelet transform, Fourier transform and other technologies to extract frequency domain features, conducts statistical analysis, and extracts statistical features such as mean, variance, and peak value.

[0048] In this embodiment, the process of fusing different feature information to obtain the target multimodal features specifically includes: dynamically adjusting the weights of different feature information according to the actual application scenario of the target multimodal large model and the preset weight adjustment algorithm; weighted fusion of different feature information based on the weights of different feature information and the attention mechanism to obtain the target multimodal features; the above fusion process is implemented by the fusion layer, and the features of different modalities are effectively fused by using the attention mechanism, the gating mechanism or the Transformer architecture (a deep learning model), etc., to form a unified multimodal representation; specifically, the fusion layer uses the attention mechanism to weightedly fuse the features of different modalities; an adaptive weight adjustment algorithm can be used to dynamically adjust the weights of different modalities according to the actual application scenario, such as using deep learning technology to extract visual features from images, extracting operation features from sensor data, and combining these features with text features extracted from operation logs according to weights. It should be noted that before the above steps, the initial multimodal large model needs to be trained using the training set to obtain the target multimodal large model, and the target large model obtained after training has the function of feature information fusion. By setting the weights of different feature information according to the actual application scenarios and fusing the feature information of different modalities according to the weights, the obtained target multimodal features are more closely aligned with the actual application scenarios, thereby improving the flexibility of the target multimodal features.

[0049] Step S13: Generate real-time production analysis results corresponding to the target production task based on the target multimodal features, the target multimodal large model and the preset large model quantization technology; obtain different types of target decisions corresponding to the target production task according to the production analysis results; and produce products according to the target decisions.

[0050] It should be noted that the target multimodal big model in this embodiment is deployed in a cloud-edge collaborative deployment mode. The core computing is completed in the cloud, and the edge device is responsible for real-time data collection and preliminary processing to ensure low latency and high reliability. The big model provides a standardized API interface (Application Programming Interface) to facilitate integration with other production management systems to ensure seamless data exchange and decision execution. In addition, the big model can also use frameworks such as Spark and TensorFlow to accelerate large-scale data processing; use tools such as Kubernetes (a container orchestration engine) for cluster management and resource scheduling, and support private cloud, public cloud or hybrid cloud deployment; can use containerization technology (such as Docker) to ensure cross-platform consistency. In addition, the big model also takes data security and privacy protection into consideration during deployment. The big model complies with relevant laws and regulations to ensure that the system meets industry standards. In addition, the big model also integrates monitoring tools to continuously monitor system performance and can retrain the model regularly to adapt to changes in data distribution.

[0051] The target multimodal big model in this embodiment is also pre-configured with online learning capabilities, and can continuously learn from newly incoming data and gradually optimize model parameters to adapt to changes and improvements in the production line. The big model can automatically adjust model parameters based on real-time feedback from the production environment to ensure the accuracy and timeliness of the model output. In addition, the multimodal big model also includes an over-modal association loss function, which enables the model to learn the intrinsic connections between different modes. At the same time, the target multimodal big model adopts a multi-task learning strategy to simultaneously process multiple downstream tasks corresponding to the target production tasks, such as equipment failure prediction, production optimization suggestions, quality control and maintenance plans, etc., to improve the generalization ability and practicality of the model.

[0052] In this embodiment, based on the target multimodal features, and using the target multimodal large model and FP8 acceleration technology, that is, the large model quantization technology, it is possible to output real-time production status prediction and analysis, that is, the above-mentioned production analysis results, which include but are not limited to equipment health status, potential fault warning, production efficiency evaluation, etc.; according to the production analysis results, the large model can generate intelligent decision suggestions, such as optimal production plan, maintenance work arrangement, resource allocation strategy and other different types of target decisions to guide production activities; the above process may specifically include: the large model uses unsupervised learning methods such as AE (Autoencoder) or VAE (Variational Auto-Encoder) for anomaly detection, and combines clustering algorithms (such as K-means) or DBSCAN (Density-Based Spatial Clustering of Applications with Noise, a density-based clustering algorithm) for pattern recognition; time series prediction models (such as ARIMA (Autoregressive Integrated Moving Average Model), Prophet, etc.) are used to predict future trends, and RL algorithms (Reinforcement Learning) are used to predict future trends. Learning, i.e. reinforcement learning algorithm) optimizes the production process, such as adjusting production parameters to improve efficiency or reduce waste. By regularly retraining the target multimodal large model, the target multimodal large model can adapt to data changes; by continuously learning based on the newly inflowing data, the model parameters can be adaptively adjusted, ensuring the accuracy and timeliness of the model output.

[0053] It can be seen that by analyzing the multimodal data in the production process, the model of this application can predict production bottlenecks and potential problems, so as to schedule and optimize in advance and improve production efficiency; by utilizing the prediction results of the multimodal model, the system can automatically adjust the production plan and resource allocation to reduce the time and cost of manual intervention; by analyzing the multimodal data of the equipment operation status, equipment failures can be predicted in advance, reducing the risk of unexpected downtime; through the results based on multimodal data analysis, managers can obtain more comprehensive and accurate information and make better decisions.

[0054] Based on the above embodiments, this application describes the process of how to generate target decisions corresponding to target production tasks based on multimodal data. Next, this application will explain the process of training a large multimodal model. Figure 4 As shown, the embodiment of the present application discloses a process of multimodal large model training, including:

[0055] Step S21: collecting historical initial multimodal data corresponding to different historical production tasks from the target production line, and performing data cleaning operations on the historical initial multimodal data to obtain corresponding cleaned historical multimodal data.

[0056] In this embodiment, after collecting the historical initial multimodal data corresponding to different historical production tasks, it is also necessary to perform data cleaning operations on the historical initial multimodal data to remove outliers and noise in the data, and to process missing values ​​in the data to ensure consistency between different data sources. By performing data cleaning operations on the historical initial multimodal data, the accuracy of the cleaned data is guaranteed, thereby ensuring the accuracy of the subsequently acquired historical target multimodal data.

[0057] Step S22: annotating the cleaned historical multimodal data according to the data type of each modal data in the cleaned historical multimodal data, and enhancing the annotated historical multimodal data according to the data type of each modal data to obtain historical target multimodal data.

[0058] In this embodiment, the cleaned historical multimodal data is annotated according to the data type, and the annotated data is enhanced. For image, video and audio data, necessary annotation work is performed, such as target detection box, category label, etc.; for text data, semantic annotation or sentiment analysis is performed; after data annotation, image data is rotated, scaled, flipped, etc. are performed to increase data diversity; audio data is processed by adding noise, speed change, etc.; synonym replacement, sentence reorganization, etc. are performed on text data to perform data enhancement operations. By performing data annotation and data enhancement operations on historical multimodal data, historical multimodal data is made more reliable, so that the multimodal large model can better understand the data.

[0059] Step S23: construct a training set based on the historical target multimodal data, and use the training set to train the initial multimodal large model to obtain the target multimodal large model.

[0060] In this embodiment, it is necessary to divide the historical multimodal data to obtain a training set therefrom, and use the training set to train the initial multimodal large model; after the model training, it also includes: constructing a test set corresponding to the target multimodal large model according to the historical target multimodal data corresponding to the historical production tasks, and using the test set to evaluate the trained target multimodal large model. It should be noted that a part of the historical multimodal data is used to construct a validation set. After the model training is completed, the validation set is also required to verify the training results. It should be noted that the initial multimodal large model also introduces a multi-task learning framework during the training process, which is used to solve multiple related tasks at the same time and improve the generalization ability of the model. By using the validation set to verify the trained target multimodal large model, and using the test set to test the training results of the model, the accuracy of model training and evaluation is guaranteed.

[0061] In addition, during the training phase of large models, in order to improve the efficiency of model training, this embodiment also adopts knowledge distillation technology to reduce the consumption of computing resources by transferring the knowledge of large and complex models to small models; and this embodiment optimizes the model structure through model pruning technology, removes unnecessary parameters, and improves the storage and reasoning speed of the model during training; in addition, during the model training and application phase, this embodiment also uses FP8 acceleration technology to improve the speed of large model training and reasoning. By using FP8 data type on a new generation of GPU (Graphics Processing Unit, i.e., graphics processor), this embodiment can significantly improve the throughput of model training while reducing memory usage; it can be understood that FP8 technology is also applicable to model reasoning, maintaining consistency between the training and reasoning phases and avoiding additional precision correction.

[0062] It can be seen that the present application ensures the accuracy of the cleaned data by performing data cleaning operations on the historical initial multimodal data, thereby ensuring the accuracy of the historical target multimodal data subsequently acquired; by performing data annotation and data enhancement operations on the historical multimodal data, the historical multimodal data is made more reliable, so that the multimodal large model can better understand the data and improve the efficiency of model training.

[0063] See also Figure 5 As shown, the embodiment of the present invention discloses a product production device based on a large model, comprising:

[0064] A data acquisition module 11 is used to collect initial multimodal data corresponding to a target production task from a target production line, and acquire target multimodal data corresponding to the target production task based on the initial multimodal data;

[0065] An information fusion module 12 is used to extract different feature information corresponding to production data of different modes in the target multimodal data by using different neural networks, and fuse the different feature information to obtain target multimodal features;

[0066] The decision acquisition module 13 is used to generate real-time production analysis results corresponding to the target production task based on the target multimodal characteristics, the target multimodal large model and the preset large model quantization technology, obtain different types of target decisions corresponding to the target production task according to the production analysis results, and produce products according to the target decisions.

[0067] It can be seen that the present application solves the problem of not being able to use the complementary advantages of multimodal data to guide production due to processing only single modality data by collecting multimodal production data and fusing the feature information corresponding to the multimodal production data; by generating target decisions corresponding to target production tasks based on multimodal features and using target decisions to guide production tasks, the rationality of the production process is ensured, thereby improving production efficiency.

[0068] In some specific embodiments, the data acquisition module 11 may specifically include:

[0069] The data acquisition unit is used to perform a data cleaning operation on the initial multimodal data to obtain corresponding cleaned multimodal data, and perform data format conversion on the cleaned multimodal data to obtain the target multimodal data corresponding to the initial multimodal data.

[0070] In some specific embodiments, the information fusion module 12 may specifically include:

[0071] A weight adjustment unit, used to dynamically adjust the weights of different feature information according to the actual application scenario of the target multimodal large model and a preset weight adjustment algorithm;

[0072] The information fusion unit is used to perform weighted fusion of different feature information based on the weights of different feature information and an attention mechanism to obtain the target multimodal feature.

[0073] In some specific embodiments, the large model-based product production device further includes:

[0074] The model deployment module is used to deploy the target multimodal large model using a cloud-edge collaborative deployment method, and continuously monitor the target multimodal large model using a preset monitoring tool to regularly retrain the target multimodal large model.

[0075] In some specific embodiments, the large model-based product production device further includes:

[0076] A model configuration module is used to configure online learning capabilities for the target multimodal large model so that the target multimodal large model automatically adjusts model parameters according to real-time feedback information of the production environment corresponding to the target production task.

[0077] In some specific embodiments, the large model-based product production device further includes:

[0078] A data cleaning module, used for collecting historical initial multimodal data corresponding to different historical production tasks from the target production line, and performing data cleaning operations on the historical initial multimodal data to obtain corresponding cleaned historical multimodal data;

[0079] A data annotation module, used for annotating the cleaned historical multimodal data according to the data type of each modal data in the cleaned historical multimodal data, and performing data enhancement on the annotated historical multimodal data according to the data type of each modal data to obtain historical target multimodal data;

[0080] The model training module is used to construct a training set based on the historical target multimodal data, and use the training set to train the initial multimodal large model to obtain the target multimodal large model.

[0081] In some specific embodiments, the model training module further includes:

[0082] A model evaluation unit is used to construct a test set corresponding to the target multimodal large model based on the historical target multimodal data corresponding to the historical production tasks, and use the test set to evaluate the trained target multimodal large model.

[0083] Furthermore, the present application also discloses an electronic device. Figure 6 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content in the diagram cannot be regarded as any limitation on the scope of use of the present application.

[0084] Figure 6 A schematic diagram of the structure of an electronic device 20 provided in an embodiment of the present application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 is used to store a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the large model-based product production method disclosed in any of the aforementioned embodiments. In addition, the electronic device 20 in this embodiment may specifically be an electronic computer.

[0085] In this embodiment, the power supply 23 is used to provide working voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and the external device, and the communication protocol it follows is any communication protocol that can be applied to the technical solution of the present application, and is not specifically limited here; the input and output interface 25 is used to obtain external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs and is not specifically limited here.

[0086] In addition, the memory 22 as a carrier for storing resources may be a read-only memory, a random access memory, a disk or an optical disk, etc. The resources stored thereon may include an operating system 221, a computer program 222, etc., and the storage method may be temporary storage or permanent storage.

[0087] The operating system 221 is used to manage and control the hardware devices and computer programs 222 on the electronic device 20, and can be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program that can be used to complete the large model-based product production method performed by the electronic device 20 disclosed in any of the aforementioned embodiments, the computer program 222 can further include a computer program that can be used to complete other specific tasks.

[0088] Furthermore, the present application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, the aforementioned large-model-based product production method is implemented. The specific steps of the method can be referred to the corresponding contents disclosed in the aforementioned embodiments, and will not be repeated here.

[0089] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0090] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0091] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0092] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0093] The technical solution provided by the present application is introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for general technicians in this field, according to the idea of ​​the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A product production method based on a large model, characterized in that: include: Collecting initial multimodal data corresponding to a target production task from a target production line, and acquiring target multimodal data corresponding to the target production task based on the initial multimodal data; Using different neural networks to extract different feature information corresponding to production data of different modes in the target multimodal data, and fusing the different feature information to obtain target multimodal features; Based on the target multimodal features, the target multimodal large model and the preset large model quantification technology, a real-time production analysis result corresponding to the target production task is generated, different types of target decisions corresponding to the target production task are obtained according to the production analysis result, and product production is performed according to the target decision.

2. The method for producing products based on a large model according to claim 1, characterized in that: The acquiring target multimodal data corresponding to the target production task based on the initial multimodal data includes: A data cleaning operation is performed on the initial multimodal data to obtain corresponding cleaned multimodal data, and a data format conversion is performed on the cleaned multimodal data to obtain the target multimodal data corresponding to the initial multimodal data.

3. The method for producing products based on a large model according to claim 1, characterized in that: The fusing of different feature information to obtain target multimodal features includes: Dynamically adjust the weights of different feature information according to the actual application scenario of the target multimodal large model and a preset weight adjustment algorithm; Based on the weights of the different feature information and the attention mechanism, the different feature information are weightedly fused to obtain the target multimodal features.

4. The method for producing products based on a large model according to claim 1, characterized in that: Also includes: The target multimodal large model is deployed using a cloud-edge collaborative deployment method, and the target multimodal large model is continuously monitored using a preset monitoring tool to regularly retrain the target multimodal large model.

5. The method for producing products based on a large model according to claim 1, characterized in that: Also includes: An online learning capability is configured for the target multimodal large model so that the target multimodal large model automatically adjusts model parameters according to real-time feedback information of the production environment corresponding to the target production task.

6. The method for producing a product based on a large model according to any one of claims 1 to 5, characterized in that: Also includes: Collecting historical initial multimodal data corresponding to different historical production tasks from the target production line, and performing data cleaning operations on the historical initial multimodal data to obtain corresponding cleaned historical multimodal data; Annotating the cleaned historical multimodal data according to the data type of each modal data in the cleaned historical multimodal data, and enhancing the annotated historical multimodal data according to the data type of each modal data to obtain historical target multimodal data; A training set is constructed according to the historical target multimodal data, and an initial multimodal large model is trained using the training set to obtain the target multimodal large model.

7. The method for producing products based on a large model according to claim 6, characterized in that: After the initial multimodal large model is trained using the training set, the method further includes: A test set corresponding to the target multimodal large model is constructed according to the historical target multimodal data corresponding to the historical production tasks, and the trained target multimodal large model is evaluated using the test set.

8. A product production device based on a large model, characterized in that: include: A data acquisition module, used to collect initial multimodal data corresponding to a target production task from a target production line, and acquire target multimodal data corresponding to the target production task based on the initial multimodal data; An information fusion module, used to extract different feature information corresponding to production data of different modes in the target multimodal data by using different neural networks, and fuse the different feature information to obtain target multimodal features; A decision acquisition module is used to generate real-time production analysis results corresponding to the target production task based on the target multimodal features, the target multimodal large model and the preset large model quantization technology, obtain different types of target decisions corresponding to the target production task according to the production analysis results, and produce products according to the target decisions.

9. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor, configured to execute the computer program to implement the large model-based product production method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: Used to store a computer program, which, when executed by a processor, implements the large model-based product production method as described in any one of claims 1 to 7.

Images