A vehicle accident repair parts recommendation method and related apparatus

Abstract

This application discloses a method and related apparatus for recommending vehicle accident repair parts, relating to the field of artificial intelligence technology. The method includes: after a vehicle collision, acquiring multimodal collision association data including multiple types of collision association data; identifying the collision location based on the multiple types of collision association data; obtaining multiple collision location identification results; achieving comprehensive and accurate identification of the collision location; and then fusing and analyzing the multimodal collision association data and the multiple collision location identification results to obtain accurate repair parts recommendation results. Compared with manual on-site inspection to determine the repair parts list, this method requires less time and is more accurate.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a method and related apparatus for recommending vehicle accident repair parts. Background Technology

[0002] With the increase in car ownership, the frequency of traffic accidents is also gradually rising. During the repair of accident vehicles, accurately and quickly identifying the appropriate repair parts is crucial to the quality and efficiency of vehicle repair.

[0003] Currently, after a vehicle accident, the repair parts list is usually determined manually at the nearest repair service center through on-site inspection. This process is time-consuming and may result in inaccurate repair parts lists due to insufficient human experience or incomplete information. Summary of the Invention

[0004] In view of the above problems, this application provides a method and related device for recommending vehicle accident repair parts, which is faster and more accurate than manual on-site inspection to determine the repair parts list. The specific solution is as follows:

[0005] The first aspect of this application provides a method for recommending vehicle accident repair parts, including:

[0006] After a vehicle collision, multimodal collision correlation data is acquired, which includes multiple types of collision correlation data;

[0007] Collision locations are identified based on multiple types of collision association data, resulting in multiple collision location identification results.

[0008] Based on the multimodal collision association data and multiple collision location identification results, a repair parts recommendation result is obtained.

[0009] In one possible implementation, the step of identifying collision locations based on multiple types of collision association data to obtain multiple collision location identification results includes:

[0010] The collision location is identified based on each collision-related data group, and the collision location identification result corresponding to each collision-related data group is obtained. Each collision-related data group includes at least one type of collision-related data, and the collision-related data included in different collision-related data groups are not completely the same.

[0011] In one possible implementation, the step of identifying collision locations based on multiple types of collision association data to obtain multiple collision location identification results includes at least two of the following:

[0012] Based on the accident images and / or LiDAR data before and after the vehicle collision in the multimodal collision association data, the collision location is identified, and the collision location identification result is obtained.

[0013] The collision location is identified based on the vehicle motion trajectory in the multimodal collision association data, and the collision location identification result is obtained.

[0014] The collision location is identified based on the sensor signals and / or diagnostic fault codes in the multimodal collision correlation data, and the collision location identification result is obtained.

[0015] In one possible implementation, identifying the collision location based on the accident image and / or LiDAR data before and after the vehicle collision in the multimodal collision association data, and obtaining the collision location identification result, includes:

[0016] The accident images and / or LiDAR data before and after the vehicle collision in the multimodal collision association data are input into the collision location recognition model to obtain the collision location recognition result output by the collision location recognition model.

[0017] In one possible implementation, identifying the collision location based on the vehicle motion trajectory in the multimodal collision association data and obtaining the collision location identification result includes:

[0018] The direction of vehicle movement after the collision is determined based on the vehicle's trajectory.

[0019] The collision location identification result is determined based on the direction of movement of the vehicle after the collision.

[0020] In one possible implementation, identifying the collision location based on sensor signals and / or diagnostic fault codes in the multimodal collision correlation data, and obtaining the collision location identification result, includes:

[0021] If the changes in sensor signals before and after a vehicle collision meet the fault conditions, the part corresponding to the sensor signal is identified as the faulty part.

[0022] If the diagnostic fault code is included in the multimodal collision association data, the part corresponding to the diagnostic fault code is identified as the faulty part.

[0023] Based on the pre-defined component fault association relationships, determine the associated components of the faulty component;

[0024] Based on the vehicle body part structure diagram, the part regions corresponding to the faulty part and its associated parts are determined, and these part regions are identified as the collision location identification results.

[0025] In one possible implementation, based on the multimodal collision association data and multiple collision location identification results, a repair parts recommendation result is obtained, including:

[0026] The multimodal collision association data and the multiple collision location identification results are input into the multimodal large model;

[0027] Multiple guidance instructions are sequentially input into the multimodal large model, causing the multimodal large model to respond sequentially to the multiple guidance instructions to obtain the recommended repair parts results. The multiple guidance instructions include at least the weights of multiple collision location recognition results.

[0028] In one possible implementation, the method for recommending vehicle accident repair parts further includes:

[0029] Based on the multimodal collision association data and multiple collision location identification results, a target collision location is obtained, wherein the target collision location includes at least one location represented by a collision location identification result;

[0030] Based on the vehicle body parts structure diagram, determine the parts area corresponding to the recommended repair parts;

[0031] If the target collision location matches the part area corresponding to the recommended repair parts result, the recommended repair parts result is determined as the final recommended repair parts result.

[0032] In one possible implementation, the method for recommending vehicle accident repair parts further includes:

[0033] The final recommended repair parts are sent to the insurance company's electronic equipment.

[0034] In one possible implementation, the method for recommending vehicle accident repair parts further includes:

[0035] Based on the multimodal collision association data and the multiple collision location identification results, the degree of collision damage is obtained.

[0036] A second aspect of this application provides a vehicle accident repair parts recommendation device, comprising:

[0037] The data acquisition unit is used to acquire multimodal collision correlation data after a vehicle collision, wherein the multimodal collision correlation data includes multiple types of collision correlation data;

[0038] The collision location identification unit is used to identify collision locations based on multiple types of collision association data and obtain multiple collision location identification results.

[0039] The repair parts recommendation unit is used to obtain repair parts recommendation results based on the multimodal collision association data and multiple collision location identification results.

[0040] A third aspect of this application provides an electronic device, comprising at least one processor and a memory connected to the processor, wherein:

[0041] The memory is used to store computer programs;

[0042] The processor is used to execute the computer program so that the electronic device can implement the vehicle accident repair parts recommendation method of the first aspect or any implementation thereof.

[0043] The fourth aspect of this application provides a computer program product including computer-readable instructions that, when executed on an electronic device, cause the electronic device to implement the vehicle accident repair parts recommendation method described in the first aspect or any implementation thereof.

[0044] The fifth aspect of this application provides a computer storage medium carrying one or more computer programs that, when executed by an electronic device, enable the electronic device to provide a vehicle accident repair parts recommendation method as described in the first aspect or any implementation thereof.

[0045] By utilizing the above technical solution, this application provides a method and related device for recommending vehicle accident repair parts. After a vehicle collision, it acquires multimodal collision association data including multiple types of collision association data, identifies the collision location based on the multiple types of collision association data, and obtains multiple collision location identification results. This achieves comprehensive and accurate identification of the collision location. By fusing and analyzing the multimodal collision association data and the multiple collision location identification results, it can obtain accurate repair parts recommendation results. Compared with manual on-site inspection to determine the repair parts list, it requires less time and has higher accuracy. Attached Figure Description

[0046] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale.

[0047] Figure 1 A system architecture diagram provided for an embodiment of this application;

[0048] Figure 2 This is a schematic diagram of a vehicle-mounted device structure provided in an embodiment of this application;

[0049] Figure 3 This application provides a schematic diagram of a network-side device structure.

[0050] Figure 4 A flowchart illustrating a method for recommending vehicle accident repair parts provided in an embodiment of this application;

[0051] Figure 5 A schematic diagram of a multimodal large model provided in an embodiment of this application;

[0052] Figure 6 This is a schematic diagram of a vehicle accident repair parts recommendation device provided in an embodiment of this application. Detailed Implementation

[0053] The embodiments of this application are described below with reference to the accompanying drawings. The terminology used in the implementation section of this application is for explaining specific embodiments only and is not intended to limit the scope of this application.

[0054] The embodiments of this application will now be described with reference to the accompanying drawings. Those skilled in the art will recognize that, with technological advancements and the emergence of new scenarios, the technical solutions provided in the embodiments of this application are equally applicable to similar technical problems.

[0055] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms are interchangeable where appropriate; this is merely a way of distinguishing objects with the same attributes in the embodiments of this application. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, so that a process, method, system, product, or apparatus that comprises a series of elements is not necessarily limited to those elements but may include other elements not explicitly listed or inherent to those processes, methods, products, or apparatuses.

[0056] See Figure 1 , Figure 1 A schematic diagram of a system architecture is shown. The system may include an in-vehicle device 100 and a network-side device 200. The network-side device 200 may include one or more servers (…). Figure 1 (Taking a server as an example for illustration), the network-side device 200 can provide one or more vehicle-mounted devices with the vehicle accident repair parts recommendation method provided in the embodiments of this application.

[0057] In this process, after a vehicle collision, the on-board device 100 sends multimodal collision correlation data to the network-side device 200. Based on the received multimodal collision correlation data, the network-side device 200 can obtain data such as the collision location and recommended repair parts.

[0058] It should be understood that in some optional implementations, the vehicle-mounted device 100 can also complete the process of obtaining data such as collision location and repair parts recommendation results based on multimodal collision correlation data on its own, without the need for the network-side device 200 to cooperate. This application embodiment is not limited to this.

[0059] The following description Figure 1 The product form of the vehicle-mounted equipment 100;

[0060] The vehicle-mounted device 100 in this application embodiment is an electronic device in a vehicle, and this application embodiment does not impose any limitations on it.

[0061] Figure 2 A schematic diagram of an optional hardware structure for the vehicle-mounted device 100 is shown.

[0062] refer to Figure 2 As shown, the vehicle-mounted device 100 may include a radio frequency unit 110, a memory 120, an input unit 130, a display unit 140, a camera 150 (optional), an audio circuit 160 (optional), a speaker 161 (optional), a microphone 162 (optional), a headphone jack 163 (optional), a processor 170, an external interface 180, a power supply 190, and other components. Those skilled in the art will understand that... Figure 2 This is merely an example of vehicle-mounted device 100 and does not constitute a limitation on vehicle-mounted device 100. It may include more or fewer components than shown, or combine certain components, or use different components.

[0063] The input unit 130 can be used to receive input digital or character information, and to generate key signal inputs related to user settings and function control of the vehicle-mounted device 100. Specifically, the input unit 130 may include a touchscreen 131 (optional) and / or other input devices 132. The touchscreen 131 can collect touch operations performed by the user on or near it (such as operations performed by the user using fingers, knuckles, styluses, or any suitable object on or near the touchscreen), and drive the corresponding connection devices according to a pre-set program. The touchscreen can detect the user's touch actions, convert the touch actions into touch signals and send them to the processor 170, and can receive and execute commands sent by the processor 170; the touch signal includes at least touch point coordinate information. The touchscreen 131 can provide an input interface and an output interface between the vehicle-mounted device 100 and the user. In addition, various types of touchscreens, such as resistive, capacitive, infrared, and surface acoustic wave, can be used. Besides the touchscreen 131, the input unit 130 may also include other input devices. Specifically, other input devices 132 may include, but are not limited to, one or more of the following: physical keyboard, function keys (such as volume control buttons, power buttons, etc.), trackball, joystick, etc.

[0064] Among them, the input device 132 can receive input data, etc.

[0065] The display unit 140 can be used to display information input by the user or information provided to the user, various menus of the in-vehicle device 100, interactive interfaces, file display, and / or playback of any multimedia file. In this embodiment, the display unit 140 can be used to display repair parts recommendation results, etc.

[0066] The memory 120 can be used to store instructions and data. The memory 120 may primarily include an instruction storage area and a data storage area. The data storage area can store various types of data, such as multimedia files and text. The instruction storage area can store software units such as operating systems, applications, and instructions required for at least one function, or subsets or extended sets thereof. It may also include non-volatile random access memory. It provides the processor 170 with hardware, software, and data resources for managing the computing device, supporting control software and applications. It is also used for storing multimedia files, as well as storing running programs and applications.

[0067] The processor 170 is the control center of the vehicle-mounted device 100. It connects various parts of the vehicle-mounted device 100 via various interfaces and lines. By running or executing instructions stored in the memory 120 and calling data stored in the memory 120, it performs various functions and processes data of the vehicle-mounted device 100, thereby providing overall control of the vehicle-mounted device 100. Optionally, the processor 170 may include one or more processing units; preferably, the processor 170 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 170. In some embodiments, the processor and memory can be implemented on a single chip; in some embodiments, they can also be implemented separately on independent chips. The processor 170 can also be used to generate corresponding operation control signals, send them to the corresponding components of the computing processing device, read and process data in the software, especially read and process data and programs in the memory 120, so that the various functional modules therein perform corresponding functions, thereby controlling the corresponding components to act according to the instructions.

[0068] The memory 120 can be used to store software code related to the vehicle accident repair parts recommendation method, and the processor 170 can execute the relevant steps of the vehicle accident repair parts recommendation method, and can also schedule other units (such as the above-mentioned input unit 130 and display unit 140) to achieve the corresponding functions.

[0069] The radio frequency unit 110 (optional) can be used for receiving and transmitting signals during information transmission or calls. For example, it can receive downlink information from the base station and process it for the processor 170; additionally, it can transmit uplink data to the base station. Typically, the RF circuit includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low-noise amplifier (LNA), a duplexer, etc. Furthermore, the radio frequency unit 110 can also communicate wirelessly with network devices and other devices. This wireless communication can use any communication standard or protocol, including but not limited to Global System for Mobile communication (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Long Term Evolution (LTE), email, Short Messaging Service (SMS), etc.

[0070] In this embodiment, the radio frequency unit 110 can send multimodal collision association data to the network-side device 200 and receive data such as collision location and repair part recommendation results sent by the network-side device 200.

[0071] It should be understood that the radio frequency unit 110 is optional and can be replaced with other communication interfaces, such as a network port.

[0072] The vehicle-mounted equipment 100 also includes a power supply 190 (such as a battery) that supplies power to various components. Preferably, the power supply can be logically connected to the processor 170 through a power management system, thereby enabling functions such as managing charging, discharging, and power consumption through the power management system.

[0073] The vehicle-mounted device 100 also includes an external interface 180, which can be a standard Micro USB interface or a multi-pin connector, which can be used to connect the vehicle-mounted device 100 to communicate with other devices, or to connect a charger to charge the vehicle-mounted device 100.

[0074] Although not shown, the vehicle-mounted device 100 may also include a strobe light, a wireless fidelity (WiFi) module, a Bluetooth module, sensors with various functions, etc., which will not be described in detail here. Some or all of the methods described below can be applied to, for example... Figure 2The vehicle-mounted equipment 100 shown.

[0075] The following description Figure 1 The product form of the network-side equipment 200;

[0076] Figure 3 A schematic diagram of the structure of a network-side device 200 is provided, such as... Figure 3 As shown, the network-side device 200 includes a bus 201, a processor 202, a communication interface 203, and a memory 204. The processor 202, the memory 204, and the communication interface 203 communicate with each other via the bus 201.

[0077] Bus 201 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of representation, Figure 3 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0078] The processor 202 can be any one or more of the following processors: central processing unit (CPU), graphics processing unit (GPU), microprocessor (MP), or digital signal processor (DSP).

[0079] Memory 204 may include volatile memory, such as random access memory (RAM). Memory 204 may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid state drive (SSD).

[0080] The memory 204 can be used to store software code related to the vehicle accident repair parts recommendation method, and the processor 202 can execute the steps of the vehicle accident repair parts recommendation method, and can also schedule other units to achieve the corresponding functions.

[0081] It should be understood that the network-side device 200 can be a centralized or distributed device. The processors (e.g., processor 170 and processor 202) in the vehicle-mounted device 100 and the network-side device 200 can be hardware circuits (such as application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), general-purpose processors, digital signal processors (DSPs), microprocessors or microcontrollers, etc.) or combinations of these hardware circuits. For example, the processor can be a hardware system with instruction execution capabilities, such as a CPU or DSP, or a hardware system without instruction execution capabilities, such as an ASIC or FPGA, or a combination of the aforementioned hardware systems without instruction execution capabilities and hardware systems with instruction execution capabilities.

[0082] To address the aforementioned problems, this application provides a method for recommending vehicle accident repair parts. The method for recommending vehicle accident repair parts according to this application will be described in detail below with reference to the accompanying drawings.

[0083] Reference Figure 4 , Figure 4 This is a flowchart illustrating a method for recommending vehicle accident repair parts according to an embodiment of this application. Figure 4 As shown in the embodiment of this application, a method for recommending vehicle accident repair parts may include steps 401 to 403, which are described in detail below.

[0084] 401: After a vehicle collision, acquire multimodal collision correlation data, which includes multiple types of collision correlation data;

[0085] In one possible implementation, vehicle accident repair parts recommendations are automatically triggered upon detecting a vehicle collision.

[0086] In another possible implementation, after a vehicle collision, the after-sales service order system receives a user's work order and triggers a recommendation for vehicle accident repair parts. In this case, the user can submit a work order to the after-sales service order system by calling customer service or by submitting a work order through an app. This application does not make any specific limitations.

[0087] Multimodal collision correlation data includes collision-related data acquired from multiple data sources to provide a comprehensive input dataset for subsequent analysis and processing. These data sources include: external and internal vehicle cameras and sensors, smartphones, dashcams, accident reports, insurance documents, etc.

[0088] To improve the quality and consistency of multimodal collision correlation data, preprocessing can be performed on collision-related data obtained from various data sources. Preprocessing includes data cleaning, format conversion, and data augmentation. Data cleaning removes noise and redundant data, ensuring data accuracy. Format conversion formats data from different sources to a unified standard, such as adjusting image resolution and performing word segmentation on text. Data augmentation enhances the diversity and robustness of image data.

[0089] Furthermore, in one possible implementation, the preprocessed multimodal collision association data can be stored in a distributed database system. This not only supports data scalability but also provides rapid retrieval capabilities. Relevant tag information, such as vehicle model, can be added to each data item in the distributed database system. After recommending vehicle accident repair parts, damage level and accident type can also be added to facilitate subsequent analysis and model training. Version control and access permissions for the multimodal collision association data can also be set to ensure data security and integrity, thereby improving the effectiveness and reliability of vehicle accident repair part recommendations.

[0090] The types of multimodal collision-related data include: accident images, LiDAR data, vehicle trajectory, sensor signals, and diagnostic fault codes.

[0091] 402: Identify collision locations based on multiple types of collision association data to obtain multiple collision location identification results;

[0092] All types of collision association data in multimodal collision association data can be used to identify collision locations, or only some types of collision association data can be used to identify collision locations.

[0093] Multiple collision location identification results may be identical or not entirely the same. For example, if the vehicle collision is minor, such as a slight scratch, the collision location may not be identifiable based on the vehicle's trajectory, sensor signals, and diagnostic fault codes, but it can be identified using accident images and LiDAR data. If the collision is severe, the collision location can be identified based on the vehicle's trajectory, sensor signals, diagnostic fault codes, accident images, and LiDAR data. However, using only some types of collision correlation data to identify the collision location may not be comprehensive; for example, relying solely on the vehicle's trajectory, sensor signals, and diagnostic fault codes may not identify collision locations such as paint scratches or door dents. This embodiment achieves comprehensive and accurate collision location identification results by identifying the collision location based on multiple types of collision correlation data.

[0094] 403: Based on multimodal collision association data and multiple collision location identification results, the recommended repair parts are obtained.

[0095] In one possible implementation, the target collision location is obtained based on multimodal collision association data and multiple collision location identification results. The target collision location includes the location represented by at least one collision location identification result. Then, the recommended repair parts are obtained based on the target collision location and the vehicle body part result map.

[0096] In another possible implementation, multimodal collision correlation data and multiple collision location identification results are input into a multimodal large model to obtain repair parts recommendations. A multimodal large model is a machine learning or deep learning model capable of processing and analyzing data from different types of data sources (modalities). Using a multimodal large model, the fusion and unified analysis of multimodal collision correlation data and multiple collision location identification results can be achieved.

[0097] After inputting the multimodal collision association data and multiple collision location identification results into the multimodal large model, a prompt can be input into the multimodal large model, such as "generate repair parts recommendation results based on the above data", so that the multimodal large model can directly output the repair parts recommendation results. Alternatively, multiple prompts can be input into the multimodal large model in sequence, so that the multimodal large model responds to the multiple prompts in sequence, executes the processing task corresponding to each prompt, and finally integrates multiple processing tasks to generate repair parts recommendation results. This embodiment does not make specific limitations.

[0098] The multimodal large model is pre-trained, and the training data includes repair parts recommendation results and multimodal collision correlation data.

[0099] The recommended repair parts output by the multimodal large model include at least one faulty part.

[0100] This embodiment discloses a method for recommending repair parts in vehicle accidents. After a vehicle collision, multimodal collision association data including multiple types of collision association data is acquired. The collision location is identified based on the multiple types of collision association data, resulting in multiple collision location identification results. This achieves comprehensive and accurate identification of the collision location. By fusing and analyzing the multimodal collision association data and the multiple collision location identification results, accurate repair parts recommendation results can be obtained. Compared with manual on-site inspection to determine the repair parts list, this method requires less time and is more accurate.

[0101] Step 402 in the above embodiments can be implemented in multiple ways.

[0102] In one possible implementation, multiple types of collision-related data are input into the collision location recognition model to obtain multiple collision location recognition results output by the collision location recognition model, with each collision location recognition result corresponding to a confidence level.

[0103] In another possible implementation, considering that different collision location identification methods are used for different types of collision-related data to achieve higher accuracy, multiple types of collision-related data can be grouped to obtain multiple collision-related data groups. The collision location is identified according to each collision-related data group to obtain the collision location identification result corresponding to each collision-related data group. Each collision-related data group includes at least one type of collision-related data, and the collision-related data included in different collision-related data groups are not completely the same.

[0104] For example, multimodal collision correlation data includes at least two of the following three collision correlation data groups:

[0105] The first collision-related data group includes accident images and / or lidar data before and after the vehicle collision;

[0106] The second collision-related data set includes the vehicle's trajectory;

[0107] The third collision-related data group includes sensor signals and / or diagnostic fault codes.

[0108] In another possible implementation, collision locations are identified based on multiple types of collision association data, resulting in multiple collision location identification results, including at least two of the following:

[0109] A1: Based on the accident images and / or LiDAR data before and after the vehicle collision in the multimodal collision association data, identify the collision location and obtain the collision location identification result;

[0110] A2: Identify the collision location based on the vehicle motion trajectory in the multimodal collision correlation data to obtain the collision location identification result;

[0111] A3: Identify the collision location based on sensor signals and / or diagnostic fault codes in the multimodal collision correlation data to obtain the collision location identification result.

[0112] The three collision location identification methods described above will be introduced in detail below.

[0113] For the collision location identification method corresponding to A1, one possible implementation is to input the accident image and / or the LiDAR data before and after the vehicle collision from the multimodal collision association data into the collision location identification model, and obtain the collision location identification result output by the collision location identification model.

[0114] For example, an accident image is input into a collision location recognition model, and the collision location recognition result is obtained using image recognition technology. The collision location recognition model is trained in advance using accident image samples with labeled collision locations.

[0115] For example, LiDAR data before and after a vehicle collision is input into a collision location recognition model to obtain the collision location recognition result. The collision location recognition model is pre-trained using LiDAR data samples with labeled collision locations. Before and after the collision, the LiDAR data indicates that the distance between the vehicle and the collision object is less than the warning range.

[0116] For example, accident images and LiDAR data before and after a vehicle collision from multimodal collision association data are input into a collision location recognition model to obtain collision location recognition results. The collision location recognition model is pre-trained using accident image samples and LiDAR data samples with labeled collision locations. It is understood that by integrating multiple types of collision association data, the collision location recognition model has higher recognition accuracy.

[0117] For the collision location identification method corresponding to A2, the vehicle's trajectory can be obtained by using signals acquired from the lateral and longitudinal acceleration sensors on the front and rear sides in multiple consecutive time frames before and after the collision. Based on the vehicle's trajectory, the direction of motion after the collision can be determined, and the collision location identification result can be determined based on this direction of motion.

[0118] For example, if a vehicle moves backward after a collision, it is likely that the front of the vehicle collided with the object, thus determining the collision location to be at the front of the vehicle.

[0119] If a vehicle moves forward after a collision, it may have been rear-ended, indicating that the collision occurred at the rear of the vehicle.

[0120] If a vehicle moves to the left after a collision, it is likely that the right side of the vehicle was hit, thus confirming that the collision occurred on the right side of the vehicle.

[0121] If a vehicle moves to the right after a collision, it is likely that the left side of the vehicle was hit, thus confirming the collision location as the left side of the vehicle.

[0122] Regarding the collision location identification method corresponding to A3, it should first be noted that some component malfunctions in a vehicle can be detected by sensors, such as tire pressure sensors detecting tire pressure and pressure sensors detecting whether the bumper has been impacted. For some component malfunctions, the on-board diagnostic system will report Diagnostic Trouble Codes (DTCs), such as those related to emissions, engine, and transmission systems. However, some component malfunctions (such as body shell malfunctions) will neither generate sensor signals nor report diagnostic trouble codes. To address the issue that these component malfunctions cannot be identified through sensor signal changes and diagnostic trouble codes, the collision location identification method provided in this embodiment pre-sets component malfunction associations. Based on these associations, it determines the associated components of the malfunctioning component. The component malfunction association represents the relationship between malfunctioning components that can be detected by sensors and diagnostic trouble codes and those that cannot. This association can be set based on historical data or after-sales replacement experience. Therefore, the collision location identification method corresponding to A3 includes the following steps A31-A34:

[0123] A31: If the changes in sensor signals before and after a vehicle collision meet the fault conditions, the part corresponding to the sensor signal is identified as the faulty part.

[0124] The sensor signal changes corresponding to different component failures require different fault conditions to be met. For example, the sensor signal changes corresponding to tire failures require the tire pressure signal to drop from normal to the fault threshold; the sensor signal changes corresponding to bumper failures require the pressure signal to suddenly increase from the normal value.

[0125] A32: When diagnostic fault codes are included in the multimodal collision association data, the part corresponding to the diagnostic fault code is identified as the faulty part;

[0126] The parts corresponding to the diagnostic fault codes are preset.

[0127] A33: Based on the pre-defined component fault association relationships, determine the associated components of the faulty component;

[0128] For example, if the sensor signal changes indicate that the bumper was hit and the body shell was also hit, then the body shell is a component that has a fault relationship with the bumper.

[0129] A34: Based on the vehicle body part structure diagram, determine the faulty part and the part area corresponding to the associated parts of the faulty part, and determine the part area as the collision location identification result.

[0130] The structural diagram of the vehicle body parts shows the correspondence between parts and part areas.

[0131] The above-mentioned collision location identification methods are merely examples, and this application is not limited to them.

[0132] After obtaining multiple collision location identification results, in one possible implementation, the multimodal collision association data and multiple collision location identification results are input into a multimodal large model to recommend repair parts.

[0133] In one possible implementation, after inputting multimodal collision association data and multiple collision location identification results into the multimodal large model, multiple guidance instructions are sequentially input into the multimodal large model, so that the multimodal large model responds to the multiple guidance instructions in sequence to obtain the recommended repair parts results.

[0134] Multiple guidance instructions include at least the weights of the collision location identification results. For example, let the weight of the collision location identification results corresponding to the above-mentioned accident images and / or the LiDAR data before and after the vehicle collision be the first weight, the weight of the collision location identification results corresponding to the vehicle movement trajectory be the second weight, and the weight of the collision location identification results corresponding to the sensor signals and / or diagnostic fault codes be the third weight. Then the first weight > the third weight > the second weight.

[0135] For example, after inputting multimodal collision correlation data and multiple collision location identification results into a multimodal large model, the system first inputs a guiding instruction: "Please determine the environment, weather, and accident cause based on the image data." Then, it inputs another guiding instruction: "Please set the weights of multiple collision location identification results to a1, a2, and a3 according to the input order of the collision location identification results to determine the collision location." Finally, it inputs yet another guiding instruction: "Please determine the recommended repair parts based on the input data, including environment, weather, accident cause, and collision location." Through this multi-level processing, each subsequent level obtains its result based on the previous level's processing, resulting in more accurate repair parts recommendations.

[0136] like Figure 5The schematic diagram of the multimodal large model shown indicates that the input data for the multimodal large model is divided into three categories: text, images, and guidance instructions. The text includes textual data such as LiDAR data, vehicle trajectory, sensor signals, and diagnostic fault codes from the multimodal collision correlation data. The images include accident images, etc. The text data is encoded by a text encoder, the image data is encoded by a vision encoder, and the guidance instructions are encoded by a guidance instruction encoder and then aligned to achieve the fusion analysis of multimodal data and finally output the recommended repair parts results.

[0137] Furthermore, to improve the accuracy of the repair parts recommendation results, in one possible implementation of the vehicle accident repair parts recommendation method disclosed in the above embodiments, a target collision position can also be obtained based on multimodal collision association data and multiple collision position identification results. For example, the multimodal collision association data and multiple collision position identification results are input into a multimodal large model to obtain the target collision position output by the multimodal large model. The target collision position includes the position represented by at least one collision position identification result, and the target collision position is used to verify the above repair parts recommendation results. For example, based on the vehicle body part structure diagram, the part region corresponding to the repair parts recommendation result is determined. If the target collision position matches the part region corresponding to the repair parts recommendation result, the repair parts recommendation result is determined as the final repair parts recommendation result.

[0138] Taking the target collision position output by the multimodal large model as an example, if the target collision position output by the multimodal large model does not match the part area corresponding to the repair parts recommendation result, it indicates that there is a contradiction in the output result of the multimodal large model and the result is unreliable. It is necessary to re-input the multimodal collision association data and multiple collision position recognition results into the multimodal large model for processing until the collision position output by the multimodal large model matches the part area corresponding to the repair parts recommendation result.

[0139] Furthermore, after determining the final recommended repair parts, the final recommended repair parts can be sent to the insurance company's electronic equipment.

[0140] In this context, the insurance company's electronic device can be its claims terminal, which sends the final repair parts recommendation result to the insurance company's back-end server. If the electronic device executing the vehicle accident repair parts recommendation method provided in this application embodiment can communicate with the insurance company's back-end server, the insurance company's electronic device can also be the insurance company's back-end server.

[0141] In one possible implementation, after determining the final recommended repair parts, not only can the final recommended repair parts be sent to the insurance company's electronic equipment, but also multimodal collision correlation data such as accident images can be sent to the insurance company's electronic equipment, thereby providing important evidence for insurance claims, providing users with one-stop service, and effectively improving user experience.

[0142] In one possible implementation, the degree of collision damage can also be obtained based on multimodal collision correlation data and multiple collision location identification results. For example, by inputting multimodal collision correlation data and multiple collision location identification results into a large multimodal model, the degree of collision damage output by the large multimodal model can be obtained. The degree of collision damage can be represented numerically (e.g., as a percentage) or by classification labels, such as Level 1 damage, Level 2 damage, etc., with higher levels indicating greater collision damage. This provides multi-dimensional data references for vehicle maintenance.

[0143] It should be noted that the above recommendations for repair parts, collision locations, and collision damage levels can be output together by the multimodal large model, or they can be output at different stages. This application does not impose any specific limitations on these parameters.

[0144] The above describes a method for recommending vehicle accident repair parts according to embodiments of this application. The following describes the apparatus for performing the above-described method for recommending vehicle accident repair parts.

[0145] Please see Figure 6 , Figure 6 This is a schematic diagram of a vehicle accident repair parts recommendation device provided in an embodiment of this application. Figure 6 As shown, the recommended device for vehicle accident repair parts includes:

[0146] The data acquisition unit 601 is used to acquire multimodal collision correlation data after a vehicle collision, wherein the multimodal collision correlation data includes multiple types of collision correlation data;

[0147] The collision location identification unit 602 is used to identify collision locations based on multiple types of collision association data and obtain multiple collision location identification results.

[0148] The repair parts recommendation unit 603 is used to obtain repair parts recommendation results based on the multimodal collision association data and multiple collision position identification results.

[0149] In one possible implementation, the collision location identification unit 602 is specifically used to identify the collision location according to each collision association data group, and obtain the collision location identification result corresponding to each collision association data group, wherein each collision association data group includes at least one type of collision association data, and the collision association data included in different collision association data groups are not completely the same.

[0150] In one possible implementation, the collision location identification unit 602 includes at least two of the following:

[0151] The first identification subunit is used to identify the collision location based on the accident image and / or LiDAR data before and after the vehicle collision in the multimodal collision association data, and to obtain the collision location identification result.

[0152] The second identification subunit is used to identify the collision location based on the vehicle motion trajectory in the multimodal collision association data, and obtain the collision location identification result;

[0153] The third identification subunit is used to identify the collision location based on the sensor signals and / or diagnostic fault codes in the multimodal collision association data, and obtain the collision location identification result.

[0154] In one possible implementation, the first identification subunit is specifically used to input the accident image and / or the LiDAR data before and after the vehicle collision from the multimodal collision association data into the collision location identification model to obtain the collision location identification result output by the collision location identification model.

[0155] In one possible implementation, the second identification subunit is specifically used to determine the direction of movement of the vehicle after the collision based on the vehicle's trajectory; and to determine the collision location identification result based on the direction of movement of the vehicle after the collision.

[0156] In one possible implementation, the third identification subunit is specifically used to: identify the part corresponding to the sensor signal as a faulty part when the changes in sensor signals before and after a vehicle collision meet the fault conditions; identify the part corresponding to the diagnostic fault code as a faulty part when the multimodal collision association data includes the diagnostic fault code; determine the associated parts of the faulty part according to a pre-set part fault association relationship; determine the part area corresponding to the faulty part and the associated parts of the faulty part according to the vehicle body part structure diagram, and determine the part area as the collision location identification result.

[0157] In one possible implementation, the repair parts recommendation unit 603 is specifically used to input the multimodal collision association data and multiple collision location identification results into the multimodal large model; and sequentially input multiple guidance instructions into the multimodal large model, so that the multimodal large model sequentially responds to the multiple guidance instructions to obtain the repair parts recommendation results, wherein the multiple guidance instructions include at least the weights of multiple collision location identification results.

[0158] In one possible implementation, the vehicle accident repair parts recommendation device further includes:

[0159] The verification unit is used to obtain the target collision position based on the multimodal collision association data and multiple collision position identification results, wherein the target collision position includes at least one position represented by the collision position identification result; determine the part area corresponding to the repair part recommendation result according to the vehicle body part structure diagram; and determine the repair part recommendation result as the final repair part recommendation result if the target collision position matches the part area corresponding to the repair part recommendation result.

[0160] In one possible implementation, the vehicle accident repair parts recommendation device further includes:

[0161] A sending unit is used to send the final repair parts recommendation result to the insurance company's electronic equipment.

[0162] In one possible implementation, the vehicle accident repair parts recommendation device further includes:

[0163] The damage degree detection unit is used to obtain the collision damage degree based on the multimodal collision association data and multiple collision location identification results.

[0164] This embodiment discloses a vehicle accident repair parts recommendation device. After a vehicle collision, it acquires multimodal collision association data including multiple types of collision association data, identifies the collision location based on the multiple types of collision association data, and obtains multiple collision location identification results. This achieves comprehensive and accurate identification of the collision location. By fusing and analyzing the multimodal collision association data and the multiple collision location identification results, it can obtain accurate repair parts recommendation results. Compared with manual on-site inspection to determine the repair parts list, it requires less time and has higher accuracy.

[0165] This application also provides an electronic device. It includes at least one processor and a memory connected to the processor, wherein:

[0166] The memory is used to store computer programs;

[0167] The processor is used to execute the computer program so that the electronic device can implement any of the vehicle accident repair parts recommendation methods provided in the embodiments of this application.

[0168] The electronic device provided in this embodiment can be the vehicle-mounted device 100 or the network-side device 200 in the above embodiments.

[0169] This application also provides a computer program product including computer-readable instructions, which, when executed on an electronic device, cause the electronic device to implement any of the vehicle accident repair parts recommendation methods provided in this application.

[0170] This application also provides a computer-readable storage medium carrying one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement any of the vehicle accident repair parts recommendation methods provided in this application.

[0171] It should also be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. In addition, in the device embodiment drawings provided in this application, the connection relationship between modules indicates that they have a communication connection, which can be implemented as one or more communication buses or signal lines.

[0172] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware, or it can be implemented by special-purpose hardware including application-specific integrated circuits, special-purpose CPUs, special-purpose memory, special-purpose components, etc. Generally, any function performed by a computer program can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be diverse, such as analog circuits, digital circuits, or special-purpose circuits. However, for this application, software program implementation is more often the preferred implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a readable storage medium, such as a computer floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk, or optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, training equipment, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0173] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product.

[0174] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, training device, or data center to another website, computer, training device, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device or data center that integrates one or more available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state drives (SSDs)).

Claims

1. A method for recommending vehicle accident repair parts, characterized in that, include: After a vehicle collision, multimodal collision correlation data is acquired, which includes multiple types of collision correlation data; Collision locations are identified based on multiple types of collision association data, resulting in multiple collision location identification results. Based on the multimodal collision association data and multiple collision location identification results, a repair parts recommendation result is obtained.

2. The method for recommending vehicle accident repair parts according to claim 1, characterized in that, The process of identifying collision locations based on multiple types of collision association data, resulting in multiple collision location identification results, includes: The collision location is identified based on each collision-related data group, and the collision location identification result corresponding to each collision-related data group is obtained. Each collision-related data group includes at least one type of collision-related data, and the collision-related data included in different collision-related data groups are not completely the same.

3. The method for recommending vehicle accident repair parts according to claim 1, characterized in that, The step of identifying collision locations based on multiple types of collision association data, resulting in multiple collision location identification results, includes at least two of the following: Based on the accident images and / or LiDAR data before and after the vehicle collision in the multimodal collision association data, the collision location is identified, and the collision location identification result is obtained. The collision location is identified based on the vehicle motion trajectory in the multimodal collision association data, and the collision location identification result is obtained. The collision location is identified based on the sensor signals and / or diagnostic fault codes in the multimodal collision correlation data, and the collision location identification result is obtained.

4. The method for recommending vehicle accident repair parts according to claim 3, characterized in that, The step of identifying the collision location based on the accident image and / or LiDAR data before and after the vehicle collision in the multimodal collision association data, and obtaining the collision location identification result, includes: The accident images and / or LiDAR data before and after the vehicle collision in the multimodal collision association data are input into the collision location recognition model to obtain the collision location recognition result output by the collision location recognition model.

5. The method for recommending vehicle accident repair parts according to claim 3, characterized in that, The step of identifying the collision location based on the vehicle motion trajectory in the multimodal collision association data, and obtaining the collision location identification result, includes: The direction of vehicle movement after the collision is determined based on the vehicle's trajectory. The collision location identification result is determined based on the direction of movement of the vehicle after the collision.

6. The method for recommending vehicle accident repair parts according to claim 3, characterized in that, The step of identifying the collision location based on sensor signals and / or diagnostic fault codes in the multimodal collision correlation data, and obtaining the collision location identification result, includes: If the changes in sensor signals before and after a vehicle collision meet the fault conditions, the part corresponding to the sensor signal is identified as the faulty part. If the diagnostic fault code is included in the multimodal collision association data, the part corresponding to the diagnostic fault code is identified as the faulty part. Based on the pre-defined component fault association relationships, determine the associated components of the faulty component; Based on the vehicle body part structure diagram, the part regions corresponding to the faulty part and its associated parts are determined, and these part regions are identified as the collision location identification results.

7. The method for recommending vehicle accident repair parts according to any one of claims 1-6, characterized in that, Based on the multimodal collision association data and multiple collision location identification results, a repair parts recommendation result is obtained, including: The multimodal collision association data and the multiple collision location identification results are input into the multimodal large model; Multiple guidance instructions are sequentially input into the multimodal large model, causing the multimodal large model to respond sequentially to the multiple guidance instructions to obtain the recommended repair parts results. The multiple guidance instructions include at least the weights of multiple collision location recognition results.

8. The method for recommending vehicle accident repair parts according to any one of claims 1-6, characterized in that, The method for recommending vehicle accident repair parts also includes: Based on the multimodal collision association data and multiple collision location identification results, a target collision location is obtained, wherein the target collision location includes at least one location represented by a collision location identification result; Based on the vehicle body parts structure diagram, determine the parts area corresponding to the recommended repair parts; If the target collision location matches the part area corresponding to the recommended repair parts result, the recommended repair parts result is determined as the final recommended repair parts result.

9. The method for recommending vehicle accident repair parts according to claim 8, characterized in that, The method for recommending vehicle accident repair parts also includes: The final recommended repair parts are sent to the insurance company's electronic equipment.

10. The method for recommending vehicle accident repair parts according to any one of claims 1-6, characterized in that, The method for recommending vehicle accident repair parts also includes: Based on the multimodal collision association data and the multiple collision location identification results, the degree of collision damage is obtained.

11. A device for recommending vehicle accident repair parts, characterized in that, include: The data acquisition unit is used to acquire multimodal collision correlation data after a vehicle collision, wherein the multimodal collision correlation data includes multiple types of collision correlation data; The collision location identification unit is used to identify collision locations based on multiple types of collision association data and obtain multiple collision location identification results. The repair parts recommendation unit is used to obtain repair parts recommendation results based on the multimodal collision association data and multiple collision location identification results.

12. An electronic device, characterized in that, It includes at least one processor and a memory connected to the processor, wherein: The memory is used to store computer programs; The processor is used to execute the computer program to enable the electronic device to implement the vehicle accident repair parts recommendation method as described in any one of claims 1 to 10.

13. A computer program product, characterized in that, Includes computer-readable instructions that, when executed on an electronic device, cause the electronic device to implement the vehicle accident repair parts recommendation method as described in any one of claims 1 to 10.

14. A computer storage medium, characterized in that, The storage medium carries one or more computer programs that, when executed by an electronic device, enable the electronic device to implement the vehicle accident repair parts recommendation method as described in any one of claims 1 to 10.

Images