Vehicle and installation part matching method and device based on multi-source data

Abstract

The invention discloses a vehicle and installation part matching method and device based on multi-source data, and relates to the technical field of vehicle assembling.The method comprises the steps that in the scene of vehicle installation part assembly, corresponding image acquisition devices and information display devices are arranged on assembly stations, the method comprises the following steps: when a vehicle arrives at an assembly station, acquiring image data about the current state of the vehicle acquired by an image acquisition device on the assembly station in real time; design data and production data of the vehicle are acquired, and the design data, the production data and the image data are preprocessed and fused to obtain multi-source fusion data; and inputting the multi-source fusion data into a pre-trained installation part assembly model to obtain a corresponding installation part assembly scheme, and displaying the installation part assembly scheme through an information display device. The problem that in the prior art, accuracy and assembling efficiency are low when the vehicle is matched with the installation part is solved.

Description

Technical Field

[0001] The present invention relates to the technical field of vehicle assembly, and particularly relates to a method and device for matching a vehicle with an installation part based on multi-source data. Background Art

[0002] With the diverse and personalized demands in the automotive market, the automotive manufacturing industry is rapidly developing towards the direction of intelligence and flexibility. The application of intelligent manufacturing technology enables the production line to simultaneously produce multiple vehicle models and configurations to meet the diverse needs of consumers. In this context, how to efficiently and accurately match vehicle models with corresponding installation parts has become a key issue in the field of automotive assembly.

[0003] The mixed-line production of multiple vehicle models means that on the same production line, vehicles of different models and configurations need to be assembled. Due to the diversity of vehicle models and configurations, there is a wide variety of installation parts required for assembly. Assembly workers need to quickly and accurately identify and install the correct parts. This poses a huge challenge to traditional assembly methods, easily leading to assembly errors and affecting product quality and production efficiency.

[0004] Currently, many automobile manufacturers adopt RFID (Radio Frequency Identification) technology to identify and track installation parts. During the assembly process, workers use RFID reading devices to identify the information of installation parts. However, this method relies on the integrity of RFID tags and manual operations. Assembly workers need to manually scan the tags, which increases the workload and may lead to missed scans or incorrect scans due to fatigue or negligence, greatly reducing the matching accuracy between vehicle models and installation parts and the assembly efficiency. Summary of the Invention

[0005] In view of this, an object of the present invention is to provide a method and device for matching a vehicle with an installation part based on multi-source data, aiming to solve the problem of low accuracy and assembly efficiency in the matching of vehicles with installation parts in the prior art.

[0006] On the one hand, the present invention proposes a method for matching a vehicle with an installation part based on multi-source data, which is applied to the scenario of installing installation parts on a vehicle. An image acquisition device and an information display device are arranged at the assembly station. The method includes: When a vehicle arrives at the assembly station, real-time acquisition of image data of the current state of the vehicle collected by the image acquisition device at the assembly station; Acquiring the design data and production data of the vehicle, and preprocessing and fusing the design data, production data, and image data to obtain multi-source fusion data; Inputting the multi-source fusion data into a pre-trained installation part assembly model to obtain a corresponding installation part assembly plan and displaying it through the information display device.

[0007] Further, in the above vehicle and installation part matching method based on multi-source data, the training process of the pre-trained installation part assembly model includes: Collect historical design data, production data, image data during assembly, and corresponding installation part assembly plans for the vehicle; Preprocess the historical design data, production data, and image data during assembly of the vehicle, and then fuse them with the corresponding installation part assembly plan to form a training data set; Build a neural network and use the training data set to perform deep learning training on the preset neural network until the loss function of the neural network tends to be stable to obtain the installation part assembly model.

[0008] Further, in the above vehicle and installation part matching method based on multi-source data, the preset neural network includes an input layer, an embedding layer, a first convolutional layer, a first pooling layer, a second convolutional layer, a second pooling layer, a feature fusion layer, a fully connected layer, and an output layer; Among them, the input layer is used to receive the design data, production data, and image data during assembly of the vehicle respectively. The first convolutional layer, the first pooling layer, the second convolutional layer, and the second pooling layer are used to extract features from the image data to obtain high-order features. The feature fusion layer is used to splice the high-order features with the combined feature vectors generated by the design data and production data through the embedding layer to obtain a fusion vector. The fully connected layer is used to compress and perform non-linear transformation on the fusion vector, so as to extract high-level abstract features related to the installation part assembly.

[0009] Further, in the above vehicle and installation part matching method based on multi-source data, after the step of inputting the multi-source fusion data into the pre-trained installation part assembly model, obtaining the corresponding installation part assembly plan and displaying it through the information display device, the following steps are further included: Obtain the feedback data of the operator during the actual assembly process for the installation part assembly plan. The feedback data at least includes the assembly problems and corresponding operation suggestions during this assembly process; Use the installation part assembly plan and the corresponding feedback data as fine-tuning corpus to fine-tune the preset installation part assembly model to obtain the final installation part assembly model.

[0010] Further, in the above vehicle and installation part matching method based on multi-source data, the step of using the installation part assembly plan and the corresponding feedback data as fine-tuning corpus to fine-tune the preset installation part assembly model to obtain the final installation part assembly model includes: Form a target training data set with the feedback data and the corresponding installation part assembly plan, freeze the first convolutional layer, the first pooling layer, the second convolutional layer, and the second pooling layer, and then input the target training data set into the installation part assembly model for fine-tuning training; After training for a preset number of rounds, if the change in the performance metric of the validation set in the target training dataset exceeds the change threshold, the second convolutional layer and the second pooling layer are unfrozen in sequence and then fine-tuned until the change in the performance metric of the validation set is lower than the change threshold, so as to obtain the final installation part assembly model.

[0011] Further, in the above vehicle and installation part matching method based on multi-source data, the design data includes the design drawings, BOM list, and configuration parameters of the vehicle; the production data includes the production plan, order information, and process flow of the vehicle.

[0012] Further, in the above vehicle and installation part matching method based on multi-source data, the installation part assembly plan includes the basic information of the installation part, a schematic diagram of the installation position of the installation part, the assembly steps of the installation part, and precautions.

[0013] Another object of the present invention is to provide a vehicle and installation part matching device based on multi-source data, which is applied to the scenario of vehicle installation part assembly. An image acquisition device and an information display device are arranged at the assembly station. The device includes: An acquisition module, configured to, when a vehicle arrives at the assembly station, acquire in real time the image data about the current state of the vehicle collected by the image acquisition device at the assembly station; A fusion module, configured to acquire the design data and production data of the vehicle, and preprocess and fuse the design data, production data, and image data to obtain multi-source fusion data; A matching module, configured to input the multi-source fusion data into a pre-trained installation part assembly model, obtain the corresponding installation part assembly plan, and display it through the information display device.

[0014] Another object of the present invention is to provide a readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the steps of the above method are implemented.

[0015] Another object of the present invention is to provide an electronic device, including a memory, a processor, and a computer program stored on the memory and running on the processor. When the processor executes the program, the steps of the above method are implemented.

[0016] In the present invention, an image acquisition device and an information display device are provided at the assembly station. When the vehicle arrives at the assembly station, image data regarding the current state of the vehicle acquired by the image acquisition device at the assembly station is obtained in real time; the design data and production data of the vehicle are obtained, and after preprocessing the design data, production data, and image data, multi-source fusion data is obtained; the multi-source fusion data is input into a pre-trained installation part assembly model to obtain a corresponding installation part assembly plan and display it through the information display device. By utilizing the efficient recognition and matching capabilities of multi-source data and the deep learning installation part assembly model, the assembly efficiency and accuracy are greatly improved. The problem of low accuracy and assembly efficiency in the prior art when matching a vehicle with an installation part is solved. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is a flowchart of a method for matching a vehicle with an installation part based on multi-source data in the first embodiment of the present invention; Figure 2 It is a structural block diagram of a device for matching a vehicle with an installation part based on multi-source data in the third embodiment of the present invention.

[0018] The following specific embodiments will further illustrate the present invention in conjunction with the above-mentioned drawings. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0019] To facilitate the understanding of the present invention, the present invention will be described more comprehensively below with reference to the relevant drawings. Several embodiments of the present invention are shown in the drawings. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein. On the contrary, these embodiments are provided to make the disclosure of the present invention more thorough and comprehensive.

[0020] It should be noted that when an element is referred to as being "fixedly provided on" another element, it can be directly on the other element or there may also be an intermediate element. When an element is considered to be "connected" to another element, it can be directly connected to the other element or there may be an intermediate element at the same time. The terms "vertical", "horizontal", "left", "right" and similar expressions used herein are only for the purpose of illustration.

[0021] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the technical field to which the present invention belongs. The terms used in the description of the present invention herein are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "and / or" used herein includes any and all combinations of one or more of the related listed items.

[0022] Embodiment 1 Please refer to Figure 1, shown is a method for matching a vehicle with a mounting part based on multi-source data in the first embodiment of the present invention, which is applied to the scenario of mounting part assembly of a vehicle. An image acquisition device and an information display device are arranged at the assembly station. The method includes steps S10 to S12.

[0023] Step S10, when the vehicle arrives at the assembly station, the image data about the current state of the vehicle collected by the image acquisition device at the assembly station is acquired in real time.

[0024] Among them, during the vehicle assembly process, each assembly station is equipped with an image acquisition device and an information display device. The image acquisition device is used to collect image data related to the vehicle, and the information display device is used to display information such as the name, number, installation position, and operation steps of the mounting part to the operator. Specifically, the image acquisition device can use a camera, and the information display device can use a tablet or a display screen. The image data contains the current state of the vehicle, so that it can be determined which assembly link the vehicle is in. That is, it can intuitively reflect information such as the appearance of the vehicle and the installation situation of components at the current moment, which is one of the important bases for subsequent analysis. For example, it can record whether certain basic components have been installed on the vehicle.

[0025] Step S11, the design data and production data of the vehicle are acquired, and after preprocessing the design data, production data, and image data, multi-source fusion data is obtained.

[0026] Among them, the system will acquire the design data and production data of the vehicle. The design data includes the design drawings, BOM list, and configuration parameters of the vehicle, that is, it includes information such as the original design specifications, dimensions, and standard installation positions of components of the vehicle. These information are descriptions of the vehicle in an ideal state. The production data records the relevant information during the actual production process of the vehicle, such as the production plan, order information, and process flow of the vehicle.

[0027] Preprocess the design data, production data, and the previously acquired image data. The purpose of preprocessing is to convert these data into a format suitable for subsequent processing, remove noise, errors, or incomplete data, and perform standardization or normalization processing on the data. For example, for image data, operations such as image enhancement, noise reduction, and feature extraction may be performed; for design data and production data, operations such as data cleaning and format conversion may be performed.

[0028] Fuse the pre - processed design data, production data, and image data to obtain multi - source fusion data. This fusion can comprehensively utilize the information from different data sources, make up for the limitations of a single data source, and provide more comprehensive and accurate data support for subsequent analysis. For example, by comparing the actual installation situation in the image data with the standard installation position in the design data and combining the component quality information in the production data, the assembly status of the vehicle can be judged more accurately.

[0029] Step S12: Input the multi - source fusion data into the pre - trained installation part assembly model to obtain the corresponding installation part assembly plan and display it through the information display device.

[0030] Among them, the multi - source fusion data is input into the pre - trained installation part assembly model. This model is trained on a large amount of historical data and can analyze the current assembly status of the vehicle based on the input multi - source fusion data and generate the corresponding installation part assembly plan.

[0031] Specifically, the installation part assembly plan includes the basic information of the installation part, the schematic diagram of the installation position of the installation part, the assembly steps and precautions of the installation part. That is, the installation part assembly plan will detail which installation parts should be installed, as well as the specific installation positions and sequences of these installation parts. Finally, the generated installation part assembly plan will be displayed to the operator through the information display device so that the operator can perform accurate assembly operations according to the plan.

[0032] In summary, in the method for matching a vehicle with installation parts based on multi - source data in the above embodiments of the present invention, by setting an image acquisition device and an information display device at the assembly station, when the vehicle arrives at the assembly station, the image data about the current state of the vehicle collected by the image acquisition device at the assembly station is obtained in real - time; the design data and production data of the vehicle are obtained, and after pre - processing the design data, production data, and image data, they are fused to obtain multi - source fusion data; the multi - source fusion data is input into the pre - trained installation part assembly model to obtain the corresponding installation part assembly plan and display it through the information display device. Utilizing the efficient recognition and matching capabilities of multi - source data and the deep - learning installation part assembly model greatly improves the assembly efficiency and accuracy. It solves the problems of low accuracy and assembly efficiency in the prior art when matching a vehicle with installation parts.

[0033] Embodiment 2 This embodiment also proposes a method for matching a vehicle with installation parts based on multi - source data. The difference between the method for matching a vehicle with installation parts based on multi - source data in this embodiment and the method for matching a vehicle with installation parts based on multi - source data in Embodiment 1 is that: The training process of the pre - trained installation part assembly model includes: Collect historical design data, production data, image data during assembly, and corresponding installation part assembly plans for the vehicle; Preprocess the historical design data, production data, and image data during assembly for the vehicle, and then fuse them with the corresponding installation part assembly plans to form a training dataset; Build a neural network and use the training dataset to perform deep learning training on the preset neural network until the loss function of the neural network stabilizes to obtain an installation part assembly model.

[0034] First, collect a large amount of historical design data, production data, image data during assembly, and corresponding installation part assembly plans for the vehicle as training samples to form a training dataset. The training dataset can be divided into a training set, a test set, and a validation set according to a preset ratio. In this dataset, the fused data serves as input features, and the corresponding installation part assembly plan serves as the target label for training the model.

[0035] Select a suitable neural network architecture. For example, a convolutional neural network (CNN) can be used to process image data, a fully connected neural network can be used to process design data and production data, or a hybrid architecture can be adopted to jointly process different types of data. The neural network consists of multiple neuron layers. Each neuron layer performs a non-linear transformation on the input data through a specific activation function to learn complex patterns in the data. Use the constructed training dataset to train the preset neural network. During the training process, the neural network makes predictions based on the input fused data, and then compares the prediction results with the target label (i.e., the corresponding installation part assembly plan) to calculate the loss function. The loss function measures the degree of difference between the prediction result and the target label. According to the value of the loss function, an optimization algorithm (such as stochastic gradient descent, Adam, etc.) is used to adjust the parameters of the neural network to gradually reduce the value of the loss function. This process is iterated continuously until the loss function stabilizes, that is, the difference between the prediction result of the model and the target label no longer changes significantly. At this time, the trained installation part assembly model is obtained.

[0036] Specifically, the preset neural network includes an input layer, an embedding layer, a first convolutional layer, a first pooling layer, a second convolutional layer, a second pooling layer, a feature fusion layer, a fully connected layer, and an output layer; Among them, the input layer is used to receive the design data, production data, and image data during assembly of the vehicle respectively. The first convolutional layer, the first pooling layer, the second convolutional layer, and the second pooling layer are used to extract features from the image data to obtain high-order features. The feature fusion layer is used to splice the high-order features with the combined feature vectors generated by the design data and production data through the embedding layer to obtain a fusion vector. The fully connected layer is used to compress and perform non-linear transformation on the fusion vector to extract high-level abstract features related to the installation part assembly.

[0037] In addition, in some alternative embodiments of the present invention, after the step of inputting the multi-source fusion data into the pre-trained installation part assembly model to obtain the corresponding installation part assembly plan and displaying it through the information display device, the following steps are further included: Obtain the feedback data of the operator during the actual assembly process for the installation part assembly plan. The feedback data at least includes the assembly problems and corresponding operation suggestions during this assembly process; Use the installation part assembly plan and the corresponding feedback data as fine-tuning corpus to fine-tune the preset installation part assembly model to obtain the final installation part assembly model.

[0038] Among them, by collecting the feedback data of the operator during the actual assembly process, the preset installation part assembly model is fine-tuned, so as to improve the accuracy and practicability of the installation part assembly plan output by the model, and obtain a better final installation part assembly model. During the actual assembly process, the operator operates according to the assembly plan output by the installation part assembly model. During this period, the system will collect their feedback data, which at least includes two aspects. Various problems encountered by the operator when operating according to the assembly plan, and improvement suggestions put forward by the operator according to their own experience and actual situation for the encountered assembly problems. Combine the original installation part assembly plan and the corresponding feedback data to form a fine-tuning corpus. This corpus reflects the differences between the plan output by the model and the actual assembly situation, as well as how to improve these plans. Use the constructed fine-tuning corpus to fine-tune the preset installation part assembly model. Fine-tuning is a training method that makes small adjustments based on an existing pre-trained model. In this process, the model will adjust its own parameters according to the assembly problems and operation suggestions in the feedback data to learn patterns and rules that are more in line with the actual assembly situation. For example, if the feedback data indicates that the installation order of a certain installation part needs to be adjusted, the model will adjust the relevant weight parameters so that when outputting the assembly plan subsequently, the installation order of this installation part can be arranged more reasonably. After multiple iterations of fine-tuning until the performance of the model reaches a satisfactory level, the final installation part assembly model is obtained. This final model can output an installation part assembly plan that better meets the actual assembly requirements, improving the efficiency and accuracy of assembly.

[0039] Further, the step of using the installation part assembly plan and the corresponding feedback data as fine-tuning corpus to fine-tune the preset installation part assembly model to obtain the final installation part assembly model includes: Form a target training data set by combining the feedback data with the corresponding installation part assembly plan, freeze the first convolutional layer, the first pooling layer, the second convolutional layer, and the second pooling layer, and then input the target training data set into the installation part assembly model for fine-tuning training; After training for a preset number of rounds, if the change in the performance metrics of the validation set in the target training dataset exceeds the change threshold, the second convolutional layer and the second pooling layer are unfrozen in sequence and then fine-tuned until the change in the performance metrics of the validation set is lower than the change threshold, so as to obtain the final installation part assembly model.

[0040] Among them, the first convolutional layer, the first pooling layer, the second convolutional layer, and the second pooling layer in the model are frozen. The convolutional layer and the pooling layer are mainly used for feature extraction of image data. In the pre-training stage, these layers have learned some general image features, such as edges, textures, etc. Freezing these layers means that the parameters of these layers will not be updated during the fine-tuning training process. The purpose of this is to avoid causing too much perturbation to the basic feature extraction ability of the model in the initial stage of fine-tuning and maintain the stability of the model. The constructed target training dataset is input into the installation part assembly model for fine-tuning training. At this stage, except for the frozen layers, the parameters of other layers of the model (such as the feature fusion layer, the fully connected layer, and the output layer) will be updated according to the training data to adapt to the new assembly requirements. After the training reaches the preset number of rounds, it is necessary to evaluate the validation set in the target training dataset and obtain the performance metrics of the validation set, such as accuracy, recall, etc. These metrics reflect the performance of the model on the validation set. By monitoring the changes in these metrics, the training effect of the model can be judged. A change threshold is set to measure the degree of change in the performance metrics of the validation set. If the change in the performance metrics of the validation set exceeds this threshold, it indicates that the model may be overfitting or underfitting in the current training state, and the training strategy of the model needs to be adjusted. When the change in the performance metrics of the validation set exceeds the threshold, the second convolutional layer and the second pooling layer are unfrozen in sequence. Unfreezing these layers means allowing their parameters to be updated in subsequent training. By gradually unfreezing these layers, the model can further learn more complex features while maintaining a certain stability, so as to improve the performance of the model. After unfreezing some layers, continue to fine-tune the model while continuously monitoring the performance metrics of the validation set. When the change in the performance metrics of the validation set is lower than the change threshold, it indicates that the performance of the model has tended to be stable, and at this time the training process ends, and the final installation part assembly model is obtained. This phased fine-tuning training strategy not only ensures the stability of the model in the initial stage of fine-tuning but also can gradually release the potential of the model when necessary, enabling the model to better adapt to the new training data. By combining the performance metrics of the validation set to dynamically adjust the training strategy, overfitting and underfitting problems can be effectively avoided, and the generalization ability of the model can be improved.

[0041] In summary, in the method for matching a vehicle with a mounting part based on multi-source data in the above embodiments of the present invention, an image acquisition device and an information display device are arranged at the assembly station. When the vehicle arrives at the assembly station, image data about the current state of the vehicle collected by the image acquisition device at the assembly station is obtained in real time; the design data and production data of the vehicle are obtained, and the design data, production data, and image data are preprocessed and then fused to obtain multi-source fusion data; the multi-source fusion data is input into a pre-trained mounting part assembly model to obtain a corresponding mounting part assembly plan and displayed through the information display device. By using multi-source data and the efficient recognition and matching capabilities of the deep learning-based mounting part assembly model, the assembly efficiency and accuracy are greatly improved. The problem of low accuracy and assembly efficiency in the prior art when matching a vehicle with a mounting part is solved.

[0042] Embodiment 3 Please refer to Figure 2 , which shows a device for matching a vehicle with a mounting part based on multi-source data proposed in the third embodiment of the present invention, applied in the scenario of mounting part assembly of a vehicle. A corresponding image acquisition device and information display device are arranged at the assembly station. The device includes: An acquisition module 100, configured to, when the vehicle arrives at the assembly station, obtain in real time image data about the current state of the vehicle collected by the image acquisition device at the assembly station; A fusion module 200, configured to obtain the design data and production data of the vehicle, and preprocess and then fuse the design data, production data, and image data to obtain multi-source fusion data; A matching module 300, configured to input the multi-source fusion data into a pre-trained mounting part assembly model to obtain a corresponding mounting part assembly plan and display it through the information display device.

[0043] Further, in the above device for matching a vehicle with a mounting part based on multi-source data, the training process of the pre-trained mounting part assembly model includes: Collect historical design data, production data, image data during assembly of the vehicle, and the corresponding mounting part assembly plan; Preprocess the historical design data, production data, and image data during assembly of the vehicle, and then fuse them with the corresponding mounting part assembly plan to form a training data set; Build a neural network and use the training data set to perform deep learning training on the preset neural network until the loss function of the neural network tends to be stable to obtain the mounting part assembly model.

[0044] Further, in the above vehicle and mounting part matching device based on multi-source data, the preset neural network includes an input layer, an embedding layer, a first convolutional layer, a first pooling layer, a second convolutional layer, a second pooling layer, a feature fusion layer, a fully connected layer, and an output layer; Among them, the input layer is used to receive the design data, production data, and image data during assembly of the vehicle respectively. The first convolutional layer, the first pooling layer, the second convolutional layer, and the second pooling layer are used to extract high-order features from the image data. The feature fusion layer is used to splice the high-order features with the combined feature vectors generated by the design data and production data through the embedding layer to obtain a fusion vector. The fully connected layer is used to compress and non-linearly transform the fusion vector, so as to extract high-level abstract features related to the assembly of the mounting part.

[0045] Further, in the above vehicle and mounting part matching device based on multi-source data, after the step of inputting the multi-source fusion data into the pre-trained mounting part assembly model to obtain the corresponding mounting part assembly plan and displaying it through the information display device, the following steps are further included: Obtain the feedback data of the operator during the actual assembly process for the mounting part assembly plan. The feedback data includes at least the assembly problems and corresponding operation suggestions during this assembly process; Use the mounting part assembly plan and the corresponding feedback data as fine-tuning corpus to fine-tune the preset mounting part assembly model to obtain the final mounting part assembly model.

[0046] Further, in the above vehicle and mounting part matching device based on multi-source data, the step of using the mounting part assembly plan and the corresponding feedback data as fine-tuning corpus to fine-tune the preset mounting part assembly model to obtain the final mounting part assembly model includes: Form a target training data set with the feedback data and the corresponding mounting part assembly plan, freeze the first convolutional layer, the first pooling layer, the second convolutional layer, and the second pooling layer, and then input the target training data set into the mounting part assembly model for fine-tuning training; After training for a preset number of rounds, if the change in the performance index of the validation set in the target training data set exceeds the change threshold, unfreeze the second convolutional layer and the second pooling layer in turn and continue fine-tuning training until the change in the performance index of the validation set is lower than the change threshold to obtain the final mounting part assembly model.

[0047] Further, in the above vehicle and mounting part matching device based on multi-source data, the design data includes the design drawings, BOM list, and configuration parameters of the vehicle; the production data includes the production plan, order information, and process flow of the vehicle.

[0048] Further, for the above vehicle and mounting part matching device based on multi-source data, the mounting part assembly plan includes basic information of the mounting part, a schematic diagram of the mounting position of the mounting part, the assembly steps of the mounting part, and precautions.

[0049] The functions or operation steps realized when the above modules are executed are substantially the same as those in the above method embodiments, and will not be elaborated here.

[0050] Embodiment 4 On the other hand, the present invention also provides a readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the steps of the method in any one of the above Embodiments 1 to 2 are realized.

[0051] Embodiment 5 On the other hand, the present invention also provides an electronic device, the electronic device includes a memory, a processor, and a computer program stored on the memory and running on the processor, and when the processor executes the program, the steps of the method in any one of the above Embodiments 1 to 2 are realized.

[0052] The technical features of each of the above embodiments can be combined arbitrarily. For the sake of concise description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered to be within the scope described in this specification.

[0053] Those skilled in the art can understand that the logic and / or steps represented in the flowchart or described in other ways herein, for example, can be considered as a definite sequence list of executable instructions for implementing logical functions, and can be specifically implemented in any computer-readable storage medium for use by an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processor, or other systems that can fetch and execute instructions from the instruction execution system, apparatus, or device), or used in combination with these instruction execution systems, apparatus, or devices. For the purposes of this specification, a "computer-readable storage medium" can be any device that can contain, store, communicate, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device.

[0054] More specific examples (a non-exhaustive list) of computer-readable storage media include the following: electrical connection parts (electronic devices) with one or more wirings, portable computer disk cartridges (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber devices, and portable compact disc read-only memory (CDROM). Additionally, the computer-readable storage media can even be paper or other suitable media on which the program can be printed, because the program can be obtained electronically, for example, by optically scanning the paper or other media, then editing, interpreting, or processing it in other suitable ways as necessary, and then storing it in a computer memory.

[0055] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, any one or a combination of the following techniques well known in the art can be used: discrete logic circuits with logic gate circuits for implementing logical functions on data signals, application-specific integrated circuits with suitable combinational logic gate circuits, programmable gate arrays (PGA), field-programmable gate arrays (FPGA), etc.

[0056] In the description of this specification, the description with reference to terms such as "one embodiment", "some embodiments", "example", "specific example", or "some examples", etc. means that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in a suitable manner in any one or more embodiments or examples.

[0057] The above-described embodiments merely represent several implementation manners of the present invention, and their descriptions are relatively specific and detailed, but should not be construed as limiting the scope of the patent of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present invention, several modifications and improvements can still be made, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention should be subject to the appended claims.

Claims

1. A vehicle and mounting part matching method based on multi-source data, characterized in that: In the scenario of assembling vehicle mounting parts, a corresponding image acquisition device and an information display device are arranged at the assembly station, and the method includes: When the vehicle arrives at the assembly station, image data on the current state of the vehicle collected by the image acquisition device at the assembly station is acquired in real time; Acquire the design data and production data of the vehicle, and fuse the design data, production data and image data after preprocessing to obtain multi-source fusion data; The multi-source fusion data is input into the pre-trained installation parts assembly model to obtain the corresponding installation parts assembly plan and display it through the information display device.

2. The vehicle and mounting part matching method based on multi-source data according to claim 1, characterized in that: The training process of the pre-trained mounting parts assembly model includes: Collect historical design data, production data, image data to be assembled, and corresponding assembly plans for mounting parts of the vehicle; The historical design data, production data, and image data of the vehicle to be assembled are pre-processed and then fused with the corresponding assembly plan of the mounting parts to form a training data set; A neural network is established and the training data set is used to perform deep learning training on the preset neural network until the loss function of the neural network tends to be stable to obtain the installation parts assembly model.

3. The vehicle and mounting part matching method based on multi-source data according to claim 2, characterized in that: The preset neural network includes input layer, embedding layer, first convolution layer, first pooling layer, second convolution layer, second pooling layer, feature fusion layer, fully connected layer and output layer; Among them, the input layer is used to receive the vehicle's design data, production data, and image data to be assembled respectively. The first convolution layer, the first pooling layer, the second convolution layer, and the second pooling layer are used to extract features from the image data to obtain high-order features. The feature fusion layer is used to concatenate the high-order features with the merged feature vectors generated by the design data and production data through the embedding layer to obtain a fusion vector. The fully connected layer is used to compress and nonlinearly transform the fusion vector, thereby extracting high-level abstract features related to the assembly of mounting parts.

4. The vehicle and mounting part matching method based on multi-source data according to claim 3, characterized in that: After the step of inputting the multi-source fusion data into the pre-trained installation part assembly model to obtain the corresponding installation part assembly plan and displaying it through the information display device, the following steps are further included: Obtaining feedback data from operators regarding the installation parts assembly plan during the actual assembly process, where the feedback data at least includes assembly problems and corresponding operation suggestions during the assembly process; The installation parts assembly scheme and the corresponding feedback data are used as fine-tuning corpus to fine-tune the preset installation parts assembly model to obtain the final installation parts assembly model.

5. The vehicle and mounting part matching method based on multi-source data according to claim 4, characterized in that: The step of using the installation part assembly scheme and the corresponding feedback data as fine-tuning corpus to fine-tune the preset installation part assembly model to obtain the final installation part assembly model includes: The feedback data and the corresponding installation parts assembly scheme are combined into a target training data set, the first convolution layer, the first pooling layer, the second convolution layer, and the second pooling layer are frozen, and then the target training data set is input into the installation parts assembly model for fine-tuning training; After the preset rounds of training, if the performance index change of the validation set in the target training data set exceeds the change threshold, the second convolutional layer and the second pooling layer are unfrozen in turn and fine-tuning training is continued until the performance index change of the validation set is lower than the change threshold to obtain the final installation parts assembly model.

6. The vehicle and mounting part matching method based on multi-source data according to claim 5, characterized in that: The design data includes the vehicle's design drawings, BOM list, and configuration parameters; the production data includes the vehicle's production plan, order information, and process flow.

7. The vehicle and mounting part matching method based on multi-source data according to claim 1, characterized in that: The installation plan of the installation parts includes basic information of the installation parts, a schematic diagram of the installation position of the installation parts, and assembly steps and precautions for the installation parts.

8. A vehicle and mounting part matching device based on multi-source data, characterized in that: In the scenario of assembling vehicle mounting parts, a corresponding image acquisition device and an information display device are arranged on the assembly station, and the device includes: An acquisition module, used to acquire, in real time, image data on the current state of the vehicle acquired by an image acquisition device at the assembly station when the vehicle arrives at the assembly station; A fusion module is used to obtain the design data and production data of the vehicle, and to fuse the design data, production data and image data after preprocessing to obtain multi-source fusion data; The matching module is used to input the multi-source fusion data into the pre-trained installation parts assembly model, obtain the corresponding installation parts assembly plan and display it through the information display device.

9. A readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

10. An electronic device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and running on the processor, wherein the processor implements the steps of the method according to any one of claims 1 to 7 when executing the program.

Images