Vehicle and installation part matching method and device based on multi-source data

Abstract

The present invention discloses a method and device for matching vehicles and mounting parts based on multi-source data, relating to the field of vehicle assembly technology. The method comprises: applying to a scenario of assembling vehicle mounting parts, disposing a corresponding image acquisition device and an information display device at an assembly station, and the method comprises: when a vehicle arrives at the assembly station, acquiring image data of the vehicle's current state captured by the image acquisition device at the assembly station in real time; acquiring the vehicle's design data and production data, and pre-processing the design data, production data, and image data to fuse them to obtain multi-source fused data; inputting the multi-source fused data into a pre-trained mounting part assembly model to obtain a corresponding mounting part assembly plan, which is then displayed through an information display device. The present invention solves the problems of low accuracy and assembly efficiency in the prior art when matching vehicles and mounting parts.

Description

Technical Field

[0001] The present invention relates to the technical field of vehicle assembly, and in particular to a method and device for matching a vehicle and an installation part based on multi-source data. Background Art

[0002] With the increasing diversification and personalization of the automotive market, the automotive manufacturing industry is rapidly developing towards intelligence and flexibility. The application of intelligent manufacturing technologies enables production lines to simultaneously produce a variety of vehicle models and configurations to meet diverse consumer needs. In this context, efficiently and accurately matching vehicle models with corresponding mounting parts has become a key issue in automotive assembly.

[0003] Mixed-line production of multiple models means that vehicles of varying models and configurations must be assembled on the same production line. This diversity of models and configurations requires a wide variety of mounting parts, requiring assemblers to quickly and accurately identify and install the correct parts. This poses a significant challenge to traditional assembly methods, easily leading to assembly errors and compromising product quality and production efficiency.

[0004] Currently, many automakers use RFID (radio frequency identification) technology to identify and track installed parts. During the assembly process, workers use RFID readers to identify part information. However, this method relies on the integrity of RFID tags and manual operation, requiring assemblers to manually scan tags, increasing their workload and potentially leading to missed or incorrect scans due to fatigue or negligence. This significantly reduces the accuracy of matching vehicle models with installed parts and reduces assembly efficiency. Summary of the Invention

[0005] In view of this, an object of the present invention is to provide a vehicle and mounting part matching method and apparatus based on multi-source data, aiming to solve the problems of low accuracy and assembly efficiency in the prior art when matching vehicles and mounting parts.

[0006] In one aspect, the present invention provides a vehicle and mounting part matching method based on multi-source data, which is applied in the scenario of vehicle mounting part assembly. A corresponding image acquisition device and information display device are arranged at the assembly station. The method includes:

[0007] When the vehicle arrives at the assembly station, the image data of the vehicle's current state captured by the image acquisition device at the assembly station is acquired in real time;

[0008] Acquire vehicle design data and production data, and pre-process the design data, production data, and image data to fuse them to obtain multi-source fused data;

[0009] The multi-source fusion data is input into the pre-trained installation parts assembly model to obtain the corresponding installation parts assembly plan and display it through the information display device.

[0010] Furthermore, in the above-mentioned vehicle and mounting part matching method based on multi-source data, the training process of the pre-trained mounting part assembly model includes:

[0011] Collect historical vehicle design data, production data, image data to be assembled, and corresponding mounting parts assembly plans;

[0012] The historical vehicle design data, production data, and image data to be assembled are pre-processed and then fused with the corresponding installation parts assembly plan to form a training data set;

[0013] A neural network is established and the training data set is used to perform deep learning training on the preset neural network until the loss function of the neural network tends to be stable to obtain the installation part assembly model.

[0014] Furthermore, in the above-mentioned vehicle and installation part matching method based on multi-source data, the preset neural network includes an input layer, an embedding layer, a first convolutional layer, a first pooling layer, a second convolutional layer, a second pooling layer, a feature fusion layer, a fully connected layer, and an output layer;

[0015] Among them, the input layer is used to receive the vehicle's design data, production data, and image data to be assembled respectively; the first convolutional layer, the first pooling layer, the second convolutional layer, and the second pooling layer are used to extract features from the image data to obtain high-order features; the feature fusion layer is used to concatenate the high-order features with the merged feature vectors generated by the embedding layer of the design data and production data to obtain a fusion vector; the fully connected layer is used to compress and nonlinearly transform the fusion vector, thereby extracting high-level abstract features related to the assembly of mounting parts.

[0016] Furthermore, the above-mentioned vehicle and mounting part matching method based on multi-source data, wherein, after the step of inputting the multi-source fusion data into the pre-trained mounting part assembly model to obtain the corresponding mounting part assembly plan and displaying it through the information display device, further includes:

[0017] Obtaining feedback data from operators regarding the actual assembly process of the installation parts assembly plan, the feedback data at least including assembly problems and corresponding operation suggestions during the assembly process;

[0018] The installation parts assembly plan and the corresponding feedback data are used as fine-tuning corpus to fine-tune the preset installation parts assembly model to obtain the final installation parts assembly model.

[0019] Furthermore, in the above-mentioned vehicle and mounting part matching method based on multi-source data, the step of using the mounting part assembly plan and the corresponding feedback data as fine-tuning corpus to fine-tune the preset mounting part assembly model to obtain the final mounting part assembly model includes:

[0020] The feedback data and the corresponding mounting parts assembly plan are combined into a target training dataset. The first convolutional layer, the first pooling layer, the second convolutional layer, and the second pooling layer are frozen. The target training dataset is then input into the mounting parts assembly model for fine-tuning training.

[0021] After the preset rounds of training, if the performance index change of the validation set in the target training data set exceeds the change threshold, the second convolutional layer and the second pooling layer are unfrozen in turn and fine-tuning training is continued until the performance index change of the validation set is lower than the change threshold to obtain the final installation part assembly model.

[0022] Furthermore, in the above-mentioned vehicle and installation parts matching method based on multi-source data, the design data includes the vehicle's design drawings, BOM list, and configuration parameters; and the production data includes the vehicle's production plan, order information, and process flow.

[0023] Furthermore, in the above-mentioned vehicle and mounting part matching method based on multi-source data, the mounting part assembly plan includes basic information of the mounting part, a schematic diagram of the mounting part installation position, and assembly steps and precautions for the mounting part.

[0024] Another object of the present invention is to provide a vehicle and mounting part matching device based on multi-source data, which is applied in the scenario of vehicle mounting part assembly. A corresponding image acquisition device and information display device are arranged at the assembly station, and the device includes:

[0025] An acquisition module is used to acquire, in real time, image data on the current state of the vehicle captured by the image acquisition device at the assembly station when the vehicle arrives at the assembly station;

[0026] A fusion module is used to obtain the design data and production data of the vehicle, and to fuse the design data, production data and image data after pre-processing to obtain multi-source fusion data;

[0027] The matching module is used to input multi-source fusion data into the pre-trained installation part assembly model, obtain the corresponding installation part assembly plan and display it through the information display device.

[0028] Another object of the present invention is to provide a readable storage medium having a computer program stored thereon, wherein the program implements the steps of the above method when executed by a processor.

[0029] Another object of the present invention is to provide an electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the steps of the above method are implemented when the processor executes the program.

[0030] The present invention arranges an image acquisition device and an information display device at the assembly station. When a vehicle arrives at the assembly station, the present invention obtains image data about the vehicle's current state captured by the image acquisition device at the assembly station in real time. The present invention also obtains the vehicle's design data and production data, pre-processes the design data, production data, and image data, and fuses them to obtain multi-source fused data. The multi-source fused data is input into a pre-trained installation part assembly model to obtain a corresponding installation part assembly plan, which is then displayed through the information display device. By utilizing the efficient recognition and matching capabilities of multi-source data and the deep learning installation part assembly model, the assembly efficiency and accuracy are greatly improved. This solves the problem of low accuracy and assembly efficiency in matching vehicles with installation parts in the prior art. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 Flowchart of a vehicle and mounting part matching method based on multi-source data in a first embodiment of the present invention;

[0032] Figure 2 4 is a structural block diagram of a vehicle and mounting part matching device based on multi-source data in a third embodiment of the present invention.

[0033] The following specific embodiments will further illustrate the present invention in conjunction with the above-mentioned drawings. DETAILED DESCRIPTION

[0034] To facilitate understanding of the present invention, the present invention will be described more fully below with reference to the accompanying drawings. The drawings illustrate several embodiments of the present invention. However, the present invention may be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided to provide a more thorough and comprehensive understanding of the present invention.

[0035] It should be noted that when an element is referred to as being "fixed to" another element, it may be directly on the other element or there may be an intermediate element. When an element is referred to as being "connected to" another element, it may be directly connected to the other element or there may be an intermediate element. The terms "vertical," "horizontal," "left," "right," and similar expressions used herein are for illustrative purposes only.

[0036] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one skilled in the art to which this invention pertains. The terms used in this specification of the present invention are for the purpose of describing specific embodiments only and are not intended to limit the present invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0037] Example 1

[0038] See also Figure 1 , shown is a vehicle and mounting part matching method based on multi-source data in the first embodiment of the present invention, which is applied in the scenario of vehicle mounting part assembly. Corresponding image acquisition devices and information display devices are arranged at the assembly station, and the method includes steps S10 to S12.

[0039] Step S10: When the vehicle arrives at the assembly station, image data on the current state of the vehicle captured by the image capture device at the assembly station is acquired in real time.

[0040] During the vehicle assembly process, each assembly station is equipped with an image acquisition device and an information display device. The image acquisition device collects vehicle-related image data, while the information display device displays information such as the name, number, installation location, and operating steps of the installed parts to the operator. Specifically, the image acquisition device can be a camera, and the information display device can be a tablet or display screen. The image data contains the vehicle's current status, allowing the identification of the vehicle's current assembly stage. This intuitively reflects the vehicle's current appearance, component installation status, and other information, providing a crucial basis for subsequent analysis. For example, it can record whether certain basic components have been installed on the vehicle.

[0041] Step S11 , obtaining design data and production data of the vehicle, and pre-processing the design data, production data and image data to fuse them to obtain multi-source fused data.

[0042] The system captures both vehicle design and production data. Design data includes the vehicle's design drawings, bill of materials (BOMs), and configuration parameters. This data includes the vehicle's original design specifications, dimensions, and standard component installation locations. This information describes the vehicle in its ideal state. Production data records information related to the actual vehicle production process, including production plans, order information, and process flows.

[0043] Preprocess the design data, production data, and previously acquired image data. The goal of preprocessing is to convert this data into a format suitable for subsequent processing, remove noise, errors, or incomplete data, and standardize or normalize the data. For example, image data may undergo image enhancement, noise reduction, and feature extraction; design and production data may undergo data cleaning and format conversion.

[0044] Multi-source fused data is generated by fusing preprocessed design data, production data, and image data. This fusion leverages information from different data sources, overcoming the limitations of a single source and providing more comprehensive and accurate data support for subsequent analysis. For example, by comparing the actual installation conditions in image data with the standard installation positions in the design data, combined with component quality information from production data, a vehicle's assembly status can be more accurately determined.

[0045] In step S12, the multi-source fusion data is input into a pre-trained installation part assembly model to obtain a corresponding installation part assembly plan and display it through an information display device.

[0046] The multi-source fusion data is fed into a pre-trained component assembly model. This model, trained on a large amount of historical data, analyzes the vehicle's current assembly state based on the input multi-source fusion data and generates a corresponding component assembly plan.

[0047] Specifically, the assembly plan includes basic information about the components, a schematic diagram of their installation locations, and assembly steps and precautions. Specifically, it details which components should be installed, along with their specific installation locations and sequence. Finally, the generated assembly plan is displayed to the operator via an information display device, allowing them to accurately perform assembly operations according to the plan.

[0048] In summary, the vehicle and mounting part matching method based on multi-source data in the above embodiment of the present invention is to set an image acquisition device and an information display device at the assembly station. When the vehicle arrives at the assembly station, the image data about the current state of the vehicle collected by the image acquisition device at the assembly station is obtained in real time; the design data and production data of the vehicle are obtained, and the design data, production data and image data are pre-processed and fused to obtain multi-source fusion data; the multi-source fusion data is input into the pre-trained mounting part assembly model to obtain the corresponding mounting part assembly plan and display it through the information display device. The efficient recognition and matching capabilities of the multi-source data and deep learning mounting part assembly model are utilized to greatly improve the assembly efficiency and accuracy. The problem of low accuracy and assembly efficiency in the prior art when matching vehicles with mounting parts is solved.

[0049] Example 2

[0050] This embodiment also proposes a vehicle and installation component matching method based on multi-source data. The vehicle and installation component matching method based on multi-source data in this embodiment differs from the vehicle and installation component matching method based on multi-source data in the first embodiment in that:

[0051] The training process of the pre-trained mounting assembly model includes:

[0052] Collect historical vehicle design data, production data, image data to be assembled, and corresponding mounting parts assembly plans;

[0053] The historical vehicle design data, production data, and image data to be assembled are pre-processed and then fused with the corresponding installation parts assembly plan to form a training data set;

[0054] A neural network is established and the training data set is used to perform deep learning training on the preset neural network until the loss function of the neural network tends to be stable to obtain the installation part assembly model.

[0055] First, a large amount of historical vehicle design data, production data, image data to be assembled, and corresponding installation parts assembly plans are collected as training samples to form a training data set. The training data set can be divided into a training set, a test set, and a validation set according to a preset ratio. In this data set, the fused data is used as input features, and the corresponding installation parts assembly plans are used as target labels for training the model.

[0056] Select an appropriate neural network architecture, such as a convolutional neural network (CNN) for processing image data, a fully connected neural network for processing design and production data, or a hybrid architecture to jointly process different types of data. A neural network consists of multiple layers of neurons, each of which uses a specific activation function to perform nonlinear transformations on the input data, thereby learning complex patterns in the data. The pre-set neural network is trained using a pre-built training dataset. During training, the neural network makes predictions based on the fused input data, then compares the predictions with the target labels (i.e., the corresponding assembly solution for the mounting parts) and calculates a loss function. The loss function measures the degree of discrepancy between the predictions and the target labels. Based on the loss function, an optimization algorithm (such as stochastic gradient descent or Adam) is used to adjust the neural network parameters to gradually reduce the loss function. This process is iterated until the loss function stabilizes, meaning that the difference between the model's predictions and the target labels no longer changes significantly. At this point, a trained assembly model for the mounting parts is obtained.

[0057] Specifically, the preset neural network includes an input layer, an embedding layer, a first convolutional layer, a first pooling layer, a second convolutional layer, a second pooling layer, a feature fusion layer, a fully connected layer, and an output layer;

[0058] Among them, the input layer is used to receive the vehicle's design data, production data, and image data to be assembled respectively; the first convolutional layer, the first pooling layer, the second convolutional layer, and the second pooling layer are used to extract features from the image data to obtain high-order features; the feature fusion layer is used to concatenate the high-order features with the merged feature vectors generated by the embedding layer of the design data and production data to obtain a fusion vector; the fully connected layer is used to compress and nonlinearly transform the fusion vector, thereby extracting high-level abstract features related to the assembly of mounting parts.

[0059] In addition, in some optional embodiments of the present invention, after the step of inputting the multi-source fusion data into the pre-trained installation part assembly model to obtain the corresponding installation part assembly plan and displaying it through the information display device, the following steps are further included:

[0060] Obtaining feedback data from operators regarding the actual assembly process of the installation parts assembly plan, the feedback data at least including assembly problems and corresponding operation suggestions during the assembly process;

[0061] The installation parts assembly plan and the corresponding feedback data are used as fine-tuning corpus to fine-tune the preset installation parts assembly model to obtain the final installation parts assembly model.

[0062] By collecting feedback from operators during the actual assembly process, the pre-set assembly model is fine-tuned to improve the accuracy and practicality of the assembly plan output by the model, resulting in a more optimized final assembly model. During the actual assembly process, operators follow the assembly plan output by the assembly model. During this process, the system collects their feedback data, which includes at least two aspects: various problems encountered by operators while following the assembly plan, and improvement suggestions based on their own experience and actual circumstances. The original assembly plan and the corresponding feedback data are combined to form a fine-tuning corpus. This corpus reflects the differences between the model's output and the actual assembly situation, as well as how to improve these solutions. The pre-set assembly model is fine-tuned using this fine-tuning corpus. Fine-tuning is a training method that makes small adjustments to an existing pre-trained model. During this process, the model adjusts its parameters based on the assembly problems and operation suggestions in the feedback data to learn patterns and patterns that better reflect actual assembly situations. For example, if feedback data indicates that the installation sequence of a certain component needs to be adjusted, the model will adjust the relevant weight parameters so that the subsequent output of the assembly plan can arrange the installation sequence of the components more reasonably. After multiple iterations of fine-tuning, until the model performance reaches a satisfactory level, the final component assembly model is obtained. This final model can output a component assembly plan that better meets actual assembly requirements, improving assembly efficiency and accuracy.

[0063] Furthermore, the step of using the mounting part assembly scheme and the corresponding feedback data as fine-tuning corpus to fine-tune the preset mounting part assembly model to obtain the final mounting part assembly model includes:

[0064] The feedback data and the corresponding mounting parts assembly plan are combined into a target training dataset. The first convolutional layer, the first pooling layer, the second convolutional layer, and the second pooling layer are frozen. The target training dataset is then input into the mounting parts assembly model for fine-tuning training.

[0065] After the preset rounds of training, if the performance index change of the validation set in the target training data set exceeds the change threshold, the second convolutional layer and the second pooling layer are unfrozen in turn and fine-tuning training is continued until the performance index change of the validation set is lower than the change threshold to obtain the final installation part assembly model.

[0066] The first convolutional layer, the first pooling layer, the second convolutional layer, and the second pooling layer in the model are frozen. Convolutional and pooling layers are primarily used for feature extraction from image data. During the pre-training phase, these layers have already learned common image features, such as edges and textures. Freezing these layers means that their parameters will not be updated during fine-tuning training. This is done to avoid excessive perturbations to the model's basic feature extraction capabilities during the initial fine-tuning phase and to maintain model stability. The constructed target training dataset is then input into the assembly model for fine-tuning. During this phase, except for the frozen layers, the parameters of other layers in the model (such as the feature fusion layer, the fully connected layer, and the output layer) are updated based on the training data to adapt to the new assembly requirements. After a preset number of training rounds, the validation set from the target training dataset is evaluated to obtain performance metrics, such as precision and recall. These metrics reflect the model's performance on the validation set. By monitoring changes in these metrics, the effectiveness of the model training can be assessed. A threshold for the degree of change in the validation set performance metrics is set to measure the degree of change. If the performance metric on the validation set changes beyond this threshold, the model may be overfitting or underfitting in its current training state, and the model training strategy needs to be adjusted. When the validation set performance metric changes beyond the threshold, the second convolutional layer and the second pooling layer are sequentially unfrozen. Unfreezing these layers allows their parameters to update during subsequent training. By gradually unfreezing these layers, the model can learn more complex features while maintaining a certain level of stability, thereby improving performance. After unfreezing some layers, the model is fine-tuned while continuously monitoring the performance metric on the validation set. When the performance metric on the validation set changes below the threshold, the model performance has stabilized, and the training process ends, resulting in the final assembly model for the fixture. This phased fine-tuning training strategy ensures model stability during the initial fine-tuning phase while gradually unlocking the model's potential when necessary, allowing the model to better adapt to new training data. By dynamically adjusting the training strategy based on the validation set performance metric, overfitting and underfitting can be effectively avoided and the model's generalization ability can be improved.

[0067] In summary, the vehicle and mounting part matching method based on multi-source data in the above embodiment of the present invention is to set an image acquisition device and an information display device at the assembly station. When the vehicle arrives at the assembly station, the image data about the current state of the vehicle collected by the image acquisition device at the assembly station is obtained in real time; the design data and production data of the vehicle are obtained, and the design data, production data and image data are pre-processed and fused to obtain multi-source fusion data; the multi-source fusion data is input into the pre-trained mounting part assembly model to obtain the corresponding mounting part assembly plan and display it through the information display device. The efficient recognition and matching capabilities of the multi-source data and deep learning mounting part assembly model are utilized to greatly improve the assembly efficiency and accuracy. The problem of low accuracy and assembly efficiency in the prior art when matching vehicles with mounting parts is solved.

[0068] Example 3

[0069] See also Figure 2 , shown is a vehicle and mounting part matching device based on multi-source data proposed in the third embodiment of the present invention, which is applied in the scenario of vehicle mounting part assembly. A corresponding image acquisition device and information display device are arranged at the assembly station, and the device includes:

[0070] The acquisition module 100 is used to acquire, in real time, image data on the current state of the vehicle captured by the image acquisition device at the assembly station when the vehicle arrives at the assembly station;

[0071] The fusion module 200 is used to obtain the design data and production data of the vehicle, and to fuse the design data, production data and image data after pre-processing to obtain multi-source fused data;

[0072] The matching module 300 is used to input the multi-source fusion data into the pre-trained installation part assembly model, obtain the corresponding installation part assembly plan and display it through the information display device.

[0073] Furthermore, in the above-mentioned vehicle and mounting part matching device based on multi-source data, the training process of the pre-trained mounting part assembly model includes:

[0074] Collect historical vehicle design data, production data, image data to be assembled, and corresponding mounting parts assembly plans;

[0075] The historical vehicle design data, production data, and image data to be assembled are pre-processed and then fused with the corresponding installation parts assembly plan to form a training data set;

[0076] A neural network is established and the training data set is used to perform deep learning training on the preset neural network until the loss function of the neural network tends to be stable to obtain the installation part assembly model.

[0077] Furthermore, in the above-mentioned vehicle and mounting part matching device based on multi-source data, the preset neural network includes an input layer, an embedding layer, a first convolutional layer, a first pooling layer, a second convolutional layer, a second pooling layer, a feature fusion layer, a fully connected layer, and an output layer;

[0078] Among them, the input layer is used to receive the vehicle's design data, production data, and image data to be assembled respectively; the first convolutional layer, the first pooling layer, the second convolutional layer, and the second pooling layer are used to extract features from the image data to obtain high-order features; the feature fusion layer is used to concatenate the high-order features with the merged feature vectors generated by the embedding layer of the design data and production data to obtain a fusion vector; the fully connected layer is used to compress and nonlinearly transform the fusion vector, thereby extracting high-level abstract features related to the assembly of mounting parts.

[0079] Furthermore, the above-mentioned vehicle and mounting part matching device based on multi-source data, wherein, after the step of inputting the multi-source fusion data into the pre-trained mounting part assembly model to obtain the corresponding mounting part assembly plan and displaying it through the information display device, further includes:

[0080] Obtaining feedback data from operators regarding the actual assembly process of the installation parts assembly plan, the feedback data at least including assembly problems and corresponding operation suggestions during the assembly process;

[0081] The installation parts assembly plan and the corresponding feedback data are used as fine-tuning corpus to fine-tune the preset installation parts assembly model to obtain the final installation parts assembly model.

[0082] Furthermore, in the above-mentioned vehicle and mounting part matching device based on multi-source data, the step of using the mounting part assembly scheme and the corresponding feedback data as fine-tuning corpus to fine-tune the preset mounting part assembly model to obtain the final mounting part assembly model includes:

[0083] The feedback data and the corresponding mounting parts assembly plan are combined into a target training dataset. The first convolutional layer, the first pooling layer, the second convolutional layer, and the second pooling layer are frozen. The target training dataset is then input into the mounting parts assembly model for fine-tuning training.

[0084] After the preset rounds of training, if the performance index change of the validation set in the target training data set exceeds the change threshold, the second convolutional layer and the second pooling layer are unfrozen in turn and fine-tuning training is continued until the performance index change of the validation set is lower than the change threshold to obtain the final installation part assembly model.

[0085] Furthermore, in the above-mentioned vehicle and mounting part matching device based on multi-source data, the design data includes the vehicle's design drawings, BOM list, and configuration parameters; and the production data includes the vehicle's production plan, order information, and process flow.

[0086] Furthermore, in the above-mentioned vehicle and mounting part matching device based on multi-source data, the mounting part assembly plan includes basic information of the mounting part, a schematic diagram of the mounting part installation position, and assembly steps and precautions for the mounting part.

[0087] The functions or operation steps implemented when the above modules are executed are substantially the same as those in the above method embodiments and will not be described in detail here.

[0088] Example 4

[0089] Another aspect of the present invention further provides a readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in any one of the above-mentioned embodiments 1 to 2.

[0090] Example 5

[0091] On the other hand, the present invention also provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the program, the steps of the method described in any one of the above-mentioned embodiments 1 to 2 are implemented.

[0092] The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0093] Those skilled in the art will appreciate that the logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable storage medium for use by an instruction execution system, apparatus, or device (e.g., a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device), or for use in conjunction with such instruction execution system, apparatus, or device. For purposes of this specification, a "computer-readable storage medium" can be any device that can contain, store, communicate, propagate, or transmit a program for use by an instruction execution system, apparatus, or device, or in conjunction with such instruction execution system, apparatus, or device.

[0094] More specific examples (a non-exhaustive list) of computer-readable storage media include the following: an electrical connection with one or more wires (electronic device), a portable computer disk cartridge (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable storage medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering, or processing in another suitable manner as necessary, and then stored in a computer memory.

[0095] It should be understood that various components of the present invention may be implemented using hardware, software, firmware, or a combination thereof. In the aforementioned embodiments, multiple steps or methods may be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one or a combination of the following technologies known in the art may be used: a discrete logic circuit having logic gate circuits for implementing logic functions on data signals, an application-specific integrated circuit having suitable combinational logic gate circuits, a programmable gate array (PGA), a field-programmable gate array (FPGA), etc.

[0096] Throughout this specification, reference to terms such as "one embodiment," "some embodiments," "examples," "specific examples," or "some examples" means that a specific feature, structure, material, or characteristic described in conjunction with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0097] The above-described embodiments merely illustrate several implementations of the present invention, and while their descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art would be able to make numerous variations and improvements without departing from the spirit of the present invention, all of which fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be determined by the appended claims.

Claims

1. A vehicle and mounting part matching method based on multi-source data, characterized in that: In the scenario of assembling vehicle mounting parts, corresponding image acquisition devices and information display devices are arranged at the assembly station. The method includes: When the vehicle arrives at the assembly station, the image data of the vehicle's current state captured by the image acquisition device at the assembly station is acquired in real time; Acquire vehicle design data and production data, and pre-process the design data, production data, and image data to obtain multi-source data; Input multi-source data into the pre-trained installation parts assembly model to obtain the corresponding installation parts assembly plan and display it through the information display device; Obtaining feedback data from operators regarding the actual assembly process of the installation parts assembly plan, the feedback data at least including assembly problems and corresponding operation suggestions during the assembly process; The installation parts assembly plan and the corresponding feedback data are used as fine-tuning corpus to fine-tune the preset installation parts assembly model to obtain the final installation parts assembly model.

2. The vehicle and mounting part matching method based on multi-source data according to claim 1, characterized in that: The training process of the pre-trained mounting assembly model includes: Collect historical vehicle design data, production data, image data to be assembled, and corresponding mounting parts assembly plans; The historical vehicle design data, production data, and image data to be assembled are pre-processed and combined with the corresponding installation parts assembly plan to form a training data set; A neural network is established and the training data set is used to perform deep learning training on the preset neural network until the loss function of the neural network tends to be stable to obtain the installation part assembly model.

3. The vehicle and mounting part matching method based on multi-source data according to claim 2, characterized in that: The preset neural network includes input layer, embedding layer, first convolution layer, first pooling layer, second convolution layer, second pooling layer, feature fusion layer, fully connected layer and output layer; Among them, the input layer is used to receive the vehicle's design data, production data, and image data to be assembled respectively; the first convolutional layer, the first pooling layer, the second convolutional layer, and the second pooling layer are used to extract features from the image data to obtain high-order features; the feature fusion layer is used to concatenate the high-order features with the merged feature vectors generated by the embedding layer of the design data and production data to obtain a fusion vector; the fully connected layer is used to compress and nonlinearly transform the fusion vector, thereby extracting high-level abstract features related to the assembly of mounting parts.

4. The vehicle and mounting part matching method based on multi-source data according to claim 1, characterized in that: The step of using the mounting part assembly scheme and the corresponding feedback data as fine-tuning corpus to fine-tune the preset mounting part assembly model to obtain the final mounting part assembly model includes: The feedback data and the corresponding mounting parts assembly plan are combined into a target training dataset. The first convolutional layer, the first pooling layer, the second convolutional layer, and the second pooling layer are frozen. The target training dataset is then input into the mounting parts assembly model for fine-tuning training. After the preset rounds of training, if the performance index change of the validation set in the target training data set exceeds the change threshold, the second convolutional layer and the second pooling layer are unfrozen in turn and fine-tuning training is continued until the performance index change of the validation set is lower than the change threshold to obtain the final installation part assembly model.

5. The vehicle and mounting part matching method based on multi-source data according to claim 4, characterized in that: The design data includes the vehicle's design drawings, BOM list, and configuration parameters; the production data includes the vehicle's production plan, order information, and process flow.

6. The vehicle and mounting part matching method based on multi-source data according to claim 1, characterized in that: The installation plan of the mounting parts includes basic information of the mounting parts, a schematic diagram of the installation position of the mounting parts, and assembly steps and precautions for the mounting parts.

7. A vehicle and mounting part matching device based on multi-source data, characterized in that: The method for matching a vehicle and a mounting part based on multi-source data according to any one of claims 1 to 6 is applied in a scenario of assembling a mounting part of a vehicle, wherein a corresponding image acquisition device and an information display device are arranged at an assembly station, and the device comprises: An acquisition module is used to acquire, in real time, image data on the current state of the vehicle captured by the image acquisition device at the assembly station when the vehicle arrives at the assembly station; A preprocessing module is used to obtain the design data and production data of the vehicle, and preprocess the design data, production data and image data to obtain multi-source data; The matching module is used to input multi-source data into the pre-trained installation part assembly model, obtain the corresponding installation part assembly plan and display it through the information display device.

8. A readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

9. An electronic device, characterized in that: The method comprises a memory, a processor, and a computer program stored in the memory and running on the processor, wherein when the processor executes the program, the steps of the method according to any one of claims 1 to 6 are implemented.

Images