Method for Reusing Road Data, Assisted Driving System, Device and Computer Equipment

Abstract

The present application relates to a method for reusing road data, an assisted driving system, a device, and a computer device. The method includes: obtaining a deviation between the installation position of a camera on a first vehicle model and the installation position of a camera on a second vehicle model, where the first vehicle model is different from the second vehicle model; obtaining a first extrinsic matrix, where the first extrinsic matrix is the extrinsic matrix of the camera on the first vehicle model; re-correcting the first extrinsic matrix according to the deviation between the installation position of the camera on the first vehicle model and the installation position of the camera on the second vehicle model to obtain a second extrinsic matrix; obtaining first image data, where the first image data is the image data captured by the camera on the first vehicle model; and performing coordinate transformation on the pixel positions in the first image data according to the first extrinsic matrix and the second extrinsic matrix to obtain second image data. By using this method, the reuse of the image data collected in the previous project can be achieved.

Description

Technical Field

[0001] The present application relates to the field of automotive electronic information technology, and particularly to a method for reusing road data, an assisted driving system, a device, and a computer device. Background Art

[0002] For the development of assisted driving software, the reliability and performance of the software need to be verified based on a large amount of actual scenario data or simulated scenario data to optimize the software performance, improve the positive trigger rate of functions, and reduce the false trigger rate of products.

[0003] The general practice in current project development is that in the early stage of functional software development, sensors, data acquisition upper computers, storage devices, etc. required are installed on the vehicle body for actual scenario data acquisition. Generally, about 50,000 - 100,000 kilometers of data in different scenarios need to be collected. Calculated at 500 kilometers per day, the acquisition cycle is about 3 - 6 months. This process generally arranges multiple road acquisition vehicles to run in parallel to shorten the road acquisition cycle. Due to differences in the installation positions of sensors between different projects, the valuable data obtained by spending a large amount of manpower and material resources can only serve a certain project. After serving a certain project, the data collected for this project cannot be reused. Summary of the Invention

[0004] Based on this, it is necessary to provide a method for reusing road data, an assisted driving system, a device, and a computer device for the above technical problems.

[0005] In a first aspect, the present application provides a method for reusing road data. Applied to an assisted driving system, the system includes a camera, and the method includes:

[0006] Obtain the deviation between the installation position of the camera on the first vehicle model and the installation position of the camera on the second vehicle model; the first vehicle model is different from the second vehicle model;

[0007] Obtain a first extrinsic matrix; the first extrinsic matrix is the extrinsic matrix of the camera on the first vehicle model;

[0008] According to the deviation between the installation position of the camera on the first vehicle model and the installation position of the camera on the second vehicle model, correct the first extrinsic matrix again to obtain a second extrinsic matrix;

[0009] Obtain first image data; the first image data is the image data captured by the camera on the first vehicle model;

[0010] According to the first extrinsic matrix and the second extrinsic matrix, perform coordinate transformation on the pixel positions in the first image data to obtain second image data.

[0011] In one embodiment, obtaining the second image data further includes:

[0012] Obtaining vehicle body data information corresponding to the first vehicle model; the vehicle body data information includes vehicle speed and direction;

[0013] determining whether a communication matrix of the first vehicle type is the same as a communication matrix of the second vehicle type;

[0014] If the communication matrix of the first vehicle model is the same as the communication matrix of the second vehicle model, the second image data and the vehicle body data information are fed back into the software system corresponding to the second vehicle model.

[0015] In one embodiment, the method further comprises:

[0016] If the communication matrix of the first vehicle type is different from the communication matrix of the second vehicle type, determining a first mapping relationship; the first mapping relationship represents a conversion relationship between message signals in the communication matrix of the first vehicle type and message signals in the communication matrix of the second vehicle type;

[0017] According to the first mapping relationship, the communication matrix of the first vehicle type is converted into the communication matrix of the second vehicle type.

[0018] In one embodiment, performing coordinate transformation on pixel positions in the first image data according to the first extrinsic parameter matrix and the second extrinsic parameter matrix to obtain the second image data includes:

[0019] Determine the first position coordinates of each corner point in the calibration plate at the imaging point in the image according to the first extrinsic parameter matrix;

[0020] Determining the second position coordinates of each corner point in the calibration plate at the imaging point in the image according to the second extrinsic parameter matrix;

[0021] Determining a coordinate conversion relationship according to the first position coordinates and the second position coordinates;

[0022] According to the coordinate transformation relationship, coordinate transformation is performed on pixel positions in the first image data to obtain second image data.

[0023] In one embodiment, the first extrinsic parameter matrix is re-corrected according to the deviation between the installation position of the camera on the first vehicle type and the installation position of the camera on the second vehicle type to obtain the second extrinsic parameter matrix, including:

[0024] Get the camera installation position on the first vehicle model;

[0025] Determine the second mapping relationship between the position of the camera installed on the first vehicle model and the first extrinsic matrix;

[0026] According to the deviation between the installation position of the camera on the first vehicle model and the installation position of the camera on the second vehicle model, and the second mapping relationship, re-correct the first extrinsic matrix to obtain the second extrinsic matrix.

[0027] In a second aspect, the present application also provides an assisted driving system. The system includes: a raw data acquisition module, an image conversion module, and a video analysis module; the image conversion module is respectively connected to the raw data acquisition module and the video analysis module;

[0028] The raw data acquisition module is used to acquire first image data; the first image data is the image data captured by the camera on the historical vehicle model;

[0029] The image conversion module is used to re-correct the first extrinsic matrix according to the deviation between the installation position of the camera on the historical vehicle model and the installation position of the camera on the actual vehicle model to obtain the second extrinsic matrix, and perform coordinate transformation on the pixel positions in the first image data according to the first extrinsic matrix and the second extrinsic matrix to obtain the second image data; the historical vehicle model is different from the actual vehicle model; the first extrinsic matrix is the extrinsic matrix of the camera on the historical vehicle model;

[0030] The video analysis module is used to identify the target object in the second image data to obtain the first identification information of the target object; the target object includes lane lines, license plates, traffic signs, and pedestrians.

[0031] In one embodiment, the system further includes: a target object identification information fusion module, a communication conversion module, and a planning and control module; the target object identification information fusion module is respectively connected to the video analysis module and the planning and control module; the communication conversion module is connected to the planning and control module;

[0032] The target object identification information fusion module is used to fuse the first identification information with the second identification information of the target object obtained by radar to obtain the third identification information of the target object;

[0033] The communication conversion module is used to determine whether the communication matrix of the historical vehicle model is the same as that of the actual vehicle model. If the communication matrix of the historical vehicle model is different from that of the actual vehicle model, it determines the first mapping relationship, and according to the first mapping relationship, converts the communication matrix of the historical vehicle model into the communication matrix of the actual vehicle model; the first mapping relationship represents the conversion relationship between the message signals in the communication matrix of the historical vehicle model and the message signals in the communication matrix of the second vehicle model.

[0034] The planning and control module is used to obtain the vehicle body data information corresponding to the second vehicle model through the communication conversion module, and perform path planning on the vehicle corresponding to the second vehicle model according to the vehicle body data information and the third identification information; the vehicle body data information includes vehicle speed and direction.

[0035] In a third aspect, the present application also provides a device for reusing road data, and the device includes:

[0036] The first acquisition module is used to acquire and determine the deviation between the installation position of the camera on the first vehicle model and the installation position of the camera on the second vehicle model; the first vehicle model is different from the second vehicle model.

[0037] The second acquisition module is used to acquire the first external parameter matrix; the first external parameter matrix is the external parameter matrix of the camera on the first vehicle model.

[0038] The external parameter matrix correction module is used to re-correct the first external parameter matrix according to the deviation between the installation position of the camera on the first vehicle model and the installation position of the camera on the second vehicle model, and obtain the second external parameter matrix.

[0039] The third acquisition module is used to acquire the first image data; the first image data is the image data captured by the camera on the first vehicle model.

[0040] The coordinate transformation module is used to perform coordinate transformation on the pixel positions in the first image data according to the first external parameter matrix and the second external parameter matrix, and obtain the second image data.

[0041] In a fourth aspect, the present application also provides a computer device. The computer device includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0042] Acquire and determine the deviation between the installation position of the camera on the first vehicle model and the installation position of the camera on the second vehicle model; the first vehicle model is different from the second vehicle model.

[0043] Acquire the first external parameter matrix; the first external parameter matrix is the external parameter matrix of the camera on the first vehicle model.

[0044] Correct the first extrinsic matrix again according to the deviation between the installation position of the camera on the first vehicle model and the installation position of the camera on the second vehicle model to obtain a second extrinsic matrix;

[0045] Obtain first image data; the first image data is the image data captured by the camera on the first vehicle model;

[0046] Perform coordinate transformation on the pixel positions in the first image data according to the first extrinsic matrix and the second extrinsic matrix to obtain second image data.

[0047] In a fifth aspect, the present application also provides a vehicle, including a roadside sensor, a communication module, a memory, a processor, and a computer program stored on the memory and executable on the processor. The roadside sensor includes a lidar, a millimeter-wave radar, and a camera. The processor is respectively connected to the processor, the roadside sensor, and the communication module. It is characterized in that when the processor executes the computer program, the following steps are implemented:

[0048] Obtain the deviation between the installation position of the camera on the first vehicle model and the installation position of the camera on the second vehicle model; the first vehicle model and the second vehicle model are different;

[0049] Obtain a first extrinsic matrix; the first extrinsic matrix is the extrinsic matrix of the camera on the first vehicle model;

[0050] Correct the first extrinsic matrix again according to the deviation between the installation position of the camera on the first vehicle model and the installation position of the camera on the second vehicle model to obtain a second extrinsic matrix;

[0051] Obtain first image data; the first image data is the image data captured by the camera on the first vehicle model;

[0052] Perform coordinate transformation on the pixel positions in the first image data according to the first extrinsic matrix and the second extrinsic matrix to obtain second image data.

[0053] The above method for reusing road data, an assisted driving system, a device, and a computer device obtain and determine the deviation between the installation position of a camera on a first vehicle model and the installation position of a camera on a second vehicle model; the first vehicle model is different from the second vehicle model, obtain a first extrinsic matrix, where the first extrinsic matrix is the extrinsic matrix of the camera on the first vehicle model, and according to the deviation between the installation position of the camera on the first vehicle model and the installation position of the camera on the second vehicle model, re-correct the first extrinsic matrix to obtain a second extrinsic matrix, obtain first image data, where the first image data is the image data captured by the camera on the first vehicle model, and according to the first extrinsic matrix and the second extrinsic matrix, perform coordinate transformation on the pixel positions in the first image data to obtain second image data, so that the second image data can be used to test the software system corresponding to the second vehicle model, solving the problem that the data collected in the previous project cannot be reused. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 It is a block diagram of the hardware structure of a terminal for the method of reusing road data in an embodiment;

[0055] Figure 2 It is a schematic flowchart of a method for reusing road data in an embodiment;

[0056] Figure 3 It is a schematic flowchart of another method for reusing road data in an embodiment;

[0057] Figure 4 It is a schematic flowchart of a method for converting image data in an embodiment;

[0058] Figure 5 It is a schematic diagram of coordinate transformation in an embodiment;

[0059] Figure 6 It is a schematic diagram of distortion correction in an embodiment;

[0060] Figure 7 It is a schematic diagram of the image data collected under the first extrinsic rectangle in an embodiment;

[0061] Figure 8 It is a schematic diagram of the image data collected under the second extrinsic rectangle in an embodiment;

[0062] Figure 9 It is a schematic diagram of an assisted driving system in an embodiment;

[0063] Figure 10 It is a schematic diagram of another assisted driving system in an embodiment;

[0064] Figure 11Schematic diagram of yet another assisted driving system in an embodiment;

[0065] Figure 12 Structural block diagram of a device for reusing road data in an embodiment. Detailed implementation manners

[0066] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0067] Unless otherwise defined, the technical terms or scientific terms involved in the present application shall have the general meanings understood by those with ordinary skills in the technical field to which the present application belongs. In the present application, words such as "a", "one", "kind", "the", "these", etc. do not indicate a limitation in quantity, and they can be singular or plural. The terms "including", "comprising", "having" and any variants thereof involved in the present application are intended to cover non-exclusive inclusion; for example, a process, method, system, product or device including a series of steps or modules (units) is not limited to the listed steps or modules (units), but may include unlisted steps or modules (units), or may include other steps or modules (units) inherent in these processes, methods, products or devices. The terms "connected", "coupled" and the like involved in the present application are not limited to physical or mechanical connections, but may include electrical connections, whether directly or indirectly connected. The term "plurality" involved in the present application refers to two or more. "And / or" describes the association relationship of associated objects and indicates that three relationships may exist. For example, "A and / or B" may represent: A exists alone, A and B exist simultaneously, and B exists alone. Usually, the character " / " indicates that the objects associated before and after are an "or" relationship. The terms "first", "second", "third", etc. involved in the present application are only used to distinguish similar objects and do not represent a specific order for the objects.

[0068] The method embodiment provided in this embodiment can be executed on a terminal, a computer or a similar computing device. For example, it runs on a terminal. Figure 1 It is a hardware structural block diagram of a terminal for a method of reusing road data provided by an embodiment of the present application. As Figure 1 shown, the terminal may include one or more ( Figure 1Only one (not shown) processor 101 and memory 102 are illustrated. Among them, the processor 101 may include, but is not limited to, a processing device such as a central processing unit (CPU), a microcontroller unit (MCU), or a field programmable gate array (FPGA). The memory 102 may include a read-only memory (ROM) and / or a random access memory (RAM). The processor 101 may perform various appropriate actions and processes according to the computer program instructions stored in the ROM or the computer program instructions loaded from the storage unit 107 into the RAM. In the RAM, various programs and data required for the operation of the terminal may also be stored. The processor 101 and the memory 102 are connected to each other through a bus 103. The input / output interface 104 is also connected to the bus 103.

[0069] Multiple components in the terminal are connected to the input / output interface 104, including: an input unit 105, such as a keyboard, a mouse, etc.; an output unit 106, such as various types of displays, speakers, etc.; a storage unit 107, such as a magnetic disk, an optical disc, etc.; and a communication unit 108, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 108 allows the terminal to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0070] Each process and treatment in the method embodiment provided in this embodiment may be executed by the processor 101. For example, in some embodiments, the method provided in this embodiment may be implemented as a computer software program, which is tangibly included in a machine-readable medium, such as the storage unit 107. In some embodiments, part or all of the computer program may be loaded and / or installed onto the terminal via the ROM and / or the communication unit 108. When the computer program is loaded into the RAM and executed by the CPU, the steps of the method provided in this embodiment may be executed.

[0071] The assisted driving system provided in the embodiments of the present application may be disposed in a vehicle and is used to control the vehicle for intelligent assisted driving. The assisted driving system may include a detection device for real-time acquisition of detection information of traffic participants and the environment around the vehicle, such as detection information of pedestrians, lane lines, traffic signs, etc.

[0072] It should be noted that the detection device in the embodiments of the present application may implement 360-degree sensor detection. The detection device may include at least one of a vision camera, a millimeter wave sensor, and a lidar sensor, and is not limited herein.

[0073] In one embodiment, as Figure 2 shown, a method for multiplexing road data is provided. The method may be applied to an assisted driving system. The system includes a camera. The method includes the following steps:

[0074] Step 201: Obtain the deviation between the installation position of the camera on the first vehicle model and the installation position of the camera on the second vehicle model; the first vehicle model and the second vehicle model are different.

[0075] Specifically, the assisted driving system in this embodiment can be set in a vehicle and is used to control the vehicle for intelligent assisted driving. The assisted driving system can include a camera, which is used to obtain the detection information of traffic participants and the environment around the vehicle in real time. For example, detection information such as pedestrians, lane lines, and traffic signs.

[0076] The first vehicle model and the second vehicle model are respectively applied to different projects. The cameras installed on the first vehicle model and the second vehicle model are the same, but the installation position of the camera on the first vehicle model is different from the installation position of the camera on the second vehicle model. Therefore, according to the installation position of the camera on the first vehicle model and the installation position of the camera on the second vehicle model, the deviation between the installation position of the camera on the first vehicle model and the installation position of the camera on the second vehicle model can be obtained.

[0077] Step 202: Obtain the first extrinsic matrix; the first extrinsic matrix is the extrinsic matrix of the camera on the first vehicle model.

[0078] It should be noted that the extrinsic matrix is used to convert the position of the target object in the world coordinate system into the position in the camera coordinate system. The extrinsic matrices corresponding to the same camera installed at different positions are different.

[0079] Step 203: According to the deviation between the installation position of the camera on the first vehicle model and the installation position of the camera on the second vehicle model, re-correct the first extrinsic matrix to obtain the second extrinsic matrix.

[0080] Specifically, the installation position of the camera on the first vehicle model is different from the installation position of the camera on the second vehicle model. Therefore, the extrinsic matrix corresponding to the camera on the first vehicle model is different from the extrinsic matrix corresponding to the camera on the second vehicle model. Further, it will cause the image data collected by the camera on the first vehicle model to be different from the image data collected by the camera on the second vehicle model. The image data collected by the camera on the first vehicle model cannot be directly used in the software system corresponding to the second vehicle model. Thus, in this embodiment, according to the deviation between the installation position of the camera on the first vehicle model and the installation position of the camera on the second vehicle model, the first extrinsic matrix is re-corrected to obtain the second extrinsic matrix, so that according to the deviation between the second extrinsic matrix and the first extrinsic matrix, the deviation between the image data captured by the camera on the first vehicle model and the image data captured by the camera on the second vehicle model can be obtained.

[0081] Step 204: Obtain the first image data; the first image data is the image data captured by the camera on the first vehicle model.

[0082] Step 205 : performing coordinate transformation on pixel positions in the first image data according to the first extrinsic parameter matrix and the second extrinsic parameter matrix to obtain second image data.

[0083] Specifically, the intrinsic parameter coefficients corresponding to the same camera are fixed, and the same camera in different installation positions will mainly lead to different extrinsic parameter coefficients, which will further cause the collected image data to be different. In this embodiment, the first extrinsic parameter matrix is the extrinsic parameter matrix corresponding to the camera at the installation position on the first vehicle model, and the second extrinsic parameter matrix is the extrinsic parameter matrix corresponding to the camera at the installation position on the second vehicle model. Therefore, according to the difference between the first extrinsic parameter matrix and the second extrinsic parameter matrix, this embodiment can perform coordinate transformation on the pixel positions in the first image data to obtain the second image data, so that the second image data can be used to test the software system corresponding to the second vehicle model.

[0084] In the above-mentioned road data reuse method, the first extrinsic parameter matrix is re-corrected according to the deviation between the installation position of the camera on the first vehicle model and the installation position of the camera on the second vehicle model to obtain the second extrinsic parameter matrix, and the pixel positions in the first image data are coordinate transformed according to the first extrinsic parameter matrix and the second extrinsic parameter matrix to obtain the second image data, so that the second image data can be used to test the software system corresponding to the second vehicle model, thereby improving the reuse rate of the image data collected in the previous project.

[0085] In one embodiment, see Figure 3 , step 205, obtaining the second image data, and then further comprising the following steps:

[0086] Step 302: Obtain vehicle body data information corresponding to the first vehicle model; the vehicle body data information includes vehicle speed and direction.

[0087] Step 304 : Determine whether the communication matrix of the first vehicle type is the same as the communication matrix of the second vehicle type.

[0088] It should be noted that the communication matrix content mainly includes the CAN node network topology diagram, the CAN node sending message signal definition list, the message signal receiving list, etc. Different product projects often have multiple models with different configurations.

[0089] Step 306 : If the communication matrix of the first vehicle model is the same as the communication matrix of the second vehicle model, the second image data and vehicle body data information are fed back into the software system corresponding to the second vehicle model.

[0090] Specifically, if the communication matrix of the first vehicle model is the same as that of the second vehicle model, the body data information corresponding to the first vehicle model can be backfed into the software system corresponding to the second vehicle model, and the body data information corresponding to the first vehicle model can be used to test the software system corresponding to the second vehicle model. The software system is an advanced driver assistance system, which includes at least one of the following: navigation and real-time traffic system, vehicle-to-everything, adaptive cruise control, lane departure warning system, lane keeping system, collision avoidance or pre-collision system, night vision system, adaptive light control, pedestrian protection system, automatic parking system, traffic sign recognition, blind spot detection, driver fatigue detection, downhill control system, and electric vehicle warning system.

[0091] It can be understood that in this embodiment, if the communication matrix of the first vehicle model is the same as that of the second vehicle model, the second image data and the body data information corresponding to the first vehicle model can be directly backfed into the software system corresponding to the second vehicle model. Thus, the second image data and the body data information corresponding to the first vehicle model can be used to test the software system corresponding to the second vehicle model, that is, the body data information corresponding to the first vehicle model can be reused, and the image data corresponding to the first vehicle model can be converted to test the software system corresponding to the second vehicle model, realizing the reuse of the image data corresponding to the first vehicle model and solving the problem that the data collected in the previous project cannot be reused.

[0092] In one embodiment, after determining whether the communication matrix of the first vehicle model is the same as that of the second vehicle model, it includes:

[0093] If the communication matrix of the first vehicle model is different from that of the second vehicle model, determine the first mapping relationship; the first mapping relationship represents the conversion relationship between the message signals in the communication matrix of the first vehicle model and the message signals in the communication matrix of the second vehicle model;

[0094] According to the first mapping relationship, convert the communication matrix of the first vehicle model into the communication matrix of the second vehicle model.

[0095] Specifically, if the communication matrix of the first vehicle model is different from that of the second vehicle model, the body data information corresponding to the first vehicle model cannot be directly backfed into the software system corresponding to the second vehicle model. Therefore, it is necessary to convert the communication matrix corresponding to the first vehicle model into the communication matrix corresponding to the second vehicle model.

[0096] It can be understood that in this embodiment, the first mapping relationship is obtained in advance according to the conversion relationship between the message signals in the communication matrix of the first vehicle model and the message signals in the communication matrix of the second vehicle model. According to the first mapping relationship, the communication matrix of the first vehicle model can be quickly and accurately converted into the communication matrix of the second vehicle model.

[0097] In another embodiment, if the communication matrix of the first vehicle model is different from that of the second vehicle model, when compiling the communication matrix corresponding to the second vehicle model, the CAN node network topology diagram of each vehicle model is drawn according to the product configuration definition table, and then the communication matrix of the second vehicle model is modified based on the project comprehensive CAN node transmission message signal definition list and the project comprehensive signal reception relationship list corresponding to the first vehicle model.

[0098] In the above manner, the communication matrix of the first vehicle model is converted into the communication matrix of the second vehicle model, so that the body data information corresponding to the first vehicle model can be applied to the software system of the second vehicle model, improving the reuse rate of the body data information corresponding to the first vehicle model.

[0099] In one embodiment, in step 205, according to the first external parameter matrix and the second external parameter matrix, coordinate transformation is performed on the pixel positions in the first image data to obtain the second image data, as Figure 4 shown, which specifically includes the following steps:

[0100] In step 402, according to the first external parameter matrix, determine the first position coordinates of the imaging points of each corner point in the calibration board in the image.

[0101] In step 404, according to the second external parameter matrix, determine the second position coordinates of the imaging points of each corner point in the calibration board in the image.

[0102] In step 406, according to the first position coordinates and the second position coordinates, determine the coordinate transformation relationship.

[0103] In step 408, according to the coordinate transformation relationship, perform coordinate transformation on the pixel positions in the first image data to obtain the second image data.

[0104] Specifically, please refer to Figure 5 and Figure 6 , the external parameter matrix describes how the real-world coordinate points fall onto the camera coordinates through rotation and translation. It is mainly expressed by the rotation matrix and the offset matrix. The internal parameter matrix describes how, after passing through the camera, it passes through the lens, forms an image through a pinhole, and is electronically converted into pixel points. The distortion matrix describes why the pixel points do not fall at the theoretical positions, such as barrel distortion. For the same camera, its internal parameter matrix and distortion matrix describe the inherent properties after the camera is produced. Different installation positions of the same camera mainly result in different external parameter coefficients, which further cause different collected image data, as Figure 7 and Figure 8 shown. Figure 7 Figure Figure 8It is a schematic diagram of the image data collected under the second external reference rectangle in an embodiment. Therefore, in this embodiment, it is first necessary to determine the position coordinates of the imaging points of each corner point in the calibration board in the image under the first external reference matrix and the second external reference matrix respectively, and determine the coordinate conversion relationship according to the above position coordinates, so as to perform coordinate conversion on the pixel coordinates in the first image data according to the coordinate conversion relationship, so that the converted image data can be used to test the software system corresponding to the second vehicle model.

[0105] Exemplarily, obtain the corner point coordinates of the calibration board in the world coordinate system and the internal reference matrix of the camera on the first vehicle model. According to the corner point coordinates in the world coordinate system and the first external reference matrix, obtain the corner point coordinates of the calibration board in the camera coordinate system on the first vehicle model. According to the corner point coordinates of the calibration board in the camera coordinate system on the first vehicle model and the internal reference matrix of the camera on the first vehicle model, obtain the first position coordinates of the imaging points of each corner point in the calibration board in the image, and then obtain the second position coordinates of the imaging points of each corner point in the calibration board in the image according to the above method.

[0106] It can be understood that in this embodiment, the first external reference matrix is the external reference matrix corresponding to the installation position of the camera on the first vehicle model, and the second external reference matrix is the external reference matrix corresponding to the installation position of the camera on the second vehicle model. Therefore, according to the difference between the first external reference matrix and the second external reference matrix in this embodiment, the pixel position in the first image data can be coordinate-transformed to obtain the second image data, so that the second image data can be used to test the software system corresponding to the second vehicle model.

[0107] In one embodiment, in step 203, according to the deviation between the installation position of the camera on the first vehicle model and the installation position of the camera on the second vehicle model, the first external reference matrix is re-corrected to obtain the second external reference matrix, which specifically includes:

[0108] Obtain the installation position of the camera on the first vehicle model;

[0109] Determine the second mapping relationship between the installation position of the camera on the first vehicle model and the first external reference matrix;

[0110] According to the deviation between the installation position of the camera on the first vehicle model and the installation position of the camera on the second vehicle model, and the second mapping relationship, the first external reference matrix is re-corrected to obtain the second external reference matrix.

[0111] Specifically, the installation positions of the cameras on the first vehicle model and the second vehicle model are the coordinates of the cameras in the world coordinate system. According to the second mapping relationship, the mapping relationship between the horizontal and vertical coordinates of the cameras in the world coordinate system and the first extrinsic matrix can be determined. The deviation between the installation position of the camera on the first vehicle model and the installation position of the camera on the second vehicle model is the coordinate deviation in the world coordinate system. Further, based on the above coordinate deviation and the second mapping relationship, the first extrinsic matrix can be re-corrected to obtain the second extrinsic matrix.

[0112] It can be understood that in this embodiment, according to the pre-determined second mapping relationship, the second extrinsic matrix can be obtained quickly and accurately, so that based on the first extrinsic matrix and the second extrinsic matrix, the coordinate transformation of the pixel positions in the first image data can be performed, and thus the software system corresponding to the second vehicle model can be tested according to the image data after coordinate transformation.

[0113] It should be understood that although the steps in the flowcharts involved in the above embodiments are displayed sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless there is a clear indication in this article, the execution of these steps has no strict order limit, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be executed alternately or alternately with at least a part of other steps or steps or stages in other steps.

[0114] In one embodiment, please refer to Figure 9 , Figure 9 which is a schematic diagram of an assisted driving system in one embodiment. As Figure 9 shown, the system includes an original data acquisition module 91, an image conversion module 92, and a video analysis module 93; the image conversion module 92 is respectively connected to the original data acquisition module 91 and the video analysis module 91;

[0115] The original data acquisition module 91 is used to acquire first image data; the first image data is the image data captured by the camera on the historical vehicle model;

[0116] The image conversion module 92 is used to correct the first external parameter matrix again according to the deviation between the installation position of the camera on the historical vehicle model and the installation position of the camera on the actual vehicle model, obtain the second external parameter matrix, and perform coordinate transformation on the pixel positions in the first image data according to the first external parameter matrix and the second external parameter matrix to obtain the second image data; the historical vehicle model is different from the actual vehicle model; the first external parameter matrix is the external parameter matrix of the camera on the historical vehicle model.

[0117] The video analysis module 93 is used to identify the target objects in the second image data to obtain the first identification information of the target objects; the target objects include lane lines, license plates, traffic signs, and pedestrians.

[0118] Specifically, the historical vehicle model is the vehicle model in the previous project, and the actual vehicle model is the vehicle model in the current project.

[0119] It can be understood that in this embodiment, the image conversion module 92 converts the image data captured by the camera on the historical vehicle model into the image data corresponding to the camera on the actual vehicle model, so that the converted image data can be applied to the actual project, realizing the reuse of the image data collected by the camera on the historical vehicle model.

[0120] In one embodiment, please refer to Figure 10 , Figure 10 which is a schematic diagram of another assisted driving system in one embodiment. As Figure 10 shown, the system further includes a target object identification information fusion module 94, a communication conversion module 95, and a planning and control module 96; the target object identification information fusion module 94 is respectively connected to the video analysis module 93 and the planning and control module 96; the communication conversion module 95 is connected to the planning and control module 96;

[0121] The target object identification information fusion module 94 is used to fuse the first identification information with the second identification information of the target object obtained by radar to obtain the third identification information of the target object;

[0122] The communication conversion module 95 is used to determine whether the communication matrix of the historical vehicle model is the same as the communication matrix of the actual vehicle model. If the communication matrix of the historical vehicle model is different from the communication matrix of the actual vehicle model, determine the first mapping relationship, and convert the communication matrix of the historical vehicle model into the communication matrix of the actual vehicle model according to the first mapping relationship; the first mapping relationship represents the conversion relationship between the message signals in the communication matrix of the historical vehicle model and the message signals in the communication matrix of the second vehicle model.

[0123] The planning and control module 96 is used to obtain the vehicle body data information corresponding to the second vehicle model through the communication conversion module, and perform path planning on the vehicle corresponding to the second vehicle model according to the vehicle body data information and the third identification information; the vehicle body data information includes vehicle speed and direction.

[0124] In one embodiment, please refer to Figure 11 , Figure 11 which is a schematic diagram of another assisted driving system in one embodiment. The system includes an image conversion module 111, a video analysis module 112, a fusion module 113, a gateway module 114, and a planning and control module 115. The image conversion module 111 is connected to the video analysis module 112, the fusion module 113 is respectively connected to the video analysis module 112 and the planning and control module 115, and the planning and control module 115 is connected to the gateway module 114;

[0125] The image conversion module 111 is an algorithm module that performs coordinate transformation on the video data sensed by the camera based on the installation position of the sensor.

[0126] Specifically, the video data is the original scene data recognized by the camera recorded through Ethernet during the scene data acquisition process. The image conversion module 111 uses FGPA to convert the video stream data collected by the same model of camera installed on different vehicle models, thereby completing the unification of the data. It should be noted that FGPA is based on hardware programming and has the characteristics of high performance and low latency in processing video data, which can meet the needs of data backfilling in the ADAS industry.

[0127] The image conversion module 111, according to different installation positions, linearly transforms the existing video scene data by translation and rotation coordinate transformation methods into image data at different installation positions. The dimension of the information elements in the equivalent scene data remains unchanged, and the information value is equivalent to the actual installation position information data after passing through the image conversion module. The specific steps are as follows:

[0128] Step1:

[0129] The image conversion module 111: sort out the differences between the installation of the platform camera and the installation of the customer vehicle model camera, where the platform camera is the camera used in the previous project and the customer vehicle model camera is the camera used in the current project.

[0130] Step2:

[0131] The image conversion module 111: based on the differences between the installation of the platform camera and the installation of the customer vehicle model camera, recheck the external parameter matrix of the platform camera.

[0132] Step3:

[0133] The image conversion module 111: based on FGPA, move and transform the image pixel coordinates in the video data captured by the platform camera to obtain the image data corresponding to the customer vehicle model camera.

[0134] Step4:

[0135] Image conversion module 111: Load the program written in step 3 into the FPGA project.

[0136] Step5:

[0137] The program runs, and the video data and CAN vehicle body data information collected by the platform camera are fed back into the customer vehicle model.

[0138] The video parsing module 112 is used to process the video data, including extracting elements such as lane lines, targets, traffic signs, etc. and their attribute values.

[0139] The fusion module 113 is an algorithm processing module for weight distribution and reconfirmation of the same element information at the same moment in the video parsing module 112 and the radar data.

[0140] Among them, the radar data is the radar perception and recognition target data recorded through the CAN network during the scene data collection process.

[0141] The gateway module 114 is a module for mapping signals based on the CAN communication matrix.

[0142] The gateway module 114 proposed in this embodiment converts the customer project communication matrix into a platform communication matrix to ensure the connection between vehicle input signals and functional output signals, specifically including the following steps:

[0143] Step1:

[0144] The gateway module 114 is used to sort out the mapping relationship table between the platform communication matrix and the customer communication matrix.

[0145] Step2:

[0146] Generate a.h file by script conversion according to the communication matrix DBC file, where the attribute information in the mapping relationship table is defined in the.h file.

[0147] Step3:

[0148] Generate a.c file through a script file using the communication matrix mapping table.

[0149] Step4:

[0150] The gateway module 114: Load the.h and.c files generated in Step2 and Step3 into the keil5 project for compilation.

[0151] Step5:

[0152] The program runs to convert the network data of the vehicle models on the platform into the protocols of the customer vehicle models.

[0153] Based on the scenario data output by the fusion module 113, the planning and control module 115 performs path planning, requests the longitudinal and lateral actuators to execute data, issues alarms, instrument displays, and other signals.

[0154] The vehicle data is the vehicle status data information recorded through the CAN network during the scenario data acquisition process. The vehicle status data information includes at least one of the following:

[0155] Acceleration, deceleration, reverse, and turning.

[0156] The vehicle data is sent to the planning and control module 115 through the gateway module 114.

[0157] Output control sends signals to the relevant actuators through the CAN network according to the signals calculated by the planning and control module 115.

[0158] In the above way, compared with rendering different driving scenarios through scenario rendering tools such as VTD, in this embodiment, the image conversion module 111 is used to perform coordinate transformation on the real road acquisition scenario data. The scenario data has rich details, a wide scenario coverage, a large amount of data, and is closest to the real driving scenario, and can verify the entire software development process more comprehensively; compared with the actual scenario data collected by the sensor installation positions of the project itself or approximate vehicle models, in this embodiment, the image conversion module 111 is used to convert the image. Compared with using approximate installation position sensor data, the video accuracy of this embodiment is higher and the backfilling effect is better. Compared with the video data obtained by road acquisition in this project, by processing the road acquisition data of the previous project through the image conversion module 111, the road acquisition link is reduced, the waste of human and material resources is avoided, the project development progress is accelerated, the reuse rate of the road acquisition data of the previous project is improved, the same scenario is applied to different projects, and the accumulation of special scenarios can be better identified, which is used as a valuable resource for the later project development. In addition, the development process of the gateway module 114 is simple, no programming skills are required, and combined with the existing script tools, only the corresponding Mapping relationship needs to be filled in the Excel table to generate.c and.h files, and the development process is efficient and accurate.

[0159] In one embodiment, as Figure 12 shown, a device for reusing road data is provided, including: a first acquisition module, a second acquisition module, a third acquisition module, an external parameter matrix correction module, and a coordinate transformation module, where:

[0160] The first acquisition module 120 is used to obtain the deviation between the installation position of the camera on the first vehicle model and the installation position of the camera on the second vehicle model; the first vehicle model is different from the second vehicle model;

[0161] The second acquisition module 121 is configured to acquire a first extrinsic parameter matrix; the first extrinsic parameter matrix is an extrinsic parameter matrix of a camera on the first vehicle model;

[0162] an extrinsic parameter matrix correction module 122, configured to re-correct the first extrinsic parameter matrix according to a deviation between an installation position of a camera on the first vehicle type and an installation position of a camera on the second vehicle type, to obtain a second extrinsic parameter matrix;

[0163] The third acquisition module 123 is configured to acquire first image data; the first image data is image data captured by a camera on the first vehicle model;

[0164] The coordinate transformation module 124 is configured to perform coordinate transformation on pixel positions in the first image data according to the first extrinsic parameter matrix and the second extrinsic parameter matrix to obtain second image data.

[0165] In one embodiment, the road data multiplexing device further includes:

[0166] Obtaining vehicle body data information corresponding to the first vehicle model; the vehicle body data information includes vehicle speed and direction;

[0167] determining whether a communication matrix of the first vehicle type is the same as a communication matrix of the second vehicle type;

[0168] If the communication matrix of the first vehicle model is the same as the communication matrix of the second vehicle model, the second image data and the vehicle body data information are fed back into the software system corresponding to the second vehicle model.

[0169] In one embodiment, the road data multiplexing device further includes:

[0170] If the communication matrix of the first vehicle type is different from the communication matrix of the second vehicle type, determining a first mapping relationship; the first mapping relationship represents a conversion relationship between message signals in the communication matrix of the first vehicle type and message signals in the communication matrix of the second vehicle type;

[0171] According to the first mapping relationship, the communication matrix of the first vehicle type is converted into the communication matrix of the second vehicle type.

[0172] In one embodiment, the coordinate transformation module 124 includes:

[0173] Determine the first position coordinates of each corner point in the calibration plate at the imaging point in the image according to the first extrinsic parameter matrix;

[0174] Determining the second position coordinates of each corner point in the calibration plate at the imaging point in the image according to the second extrinsic parameter matrix;

[0175] Determine a coordinate transformation relationship according to the first position coordinate and the second position coordinate;

[0176] Perform a coordinate transformation on the pixel positions in the first image data according to the coordinate transformation relationship to obtain second image data.

[0177] In one embodiment, the extrinsic matrix correction module 122 includes:

[0178] Obtain the position where the camera is installed on the first vehicle model;

[0179] Determine a second mapping relationship between the position where the camera is installed on the first vehicle model and the first extrinsic matrix;

[0180] Re-correct the first extrinsic matrix according to the deviation between the installation position of the camera on the first vehicle model and the installation position of the camera on the second vehicle model, and the second mapping relationship, to obtain a second extrinsic matrix.

[0181] Each module in the above road data reuse device can be implemented in whole or in part by software, hardware, and their combination. The above modules can be embedded in the processor of the computer device in hardware form or be independent of it, or can be stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to the above modules.

[0182] In one embodiment, a vehicle is provided, including a roadside sensor, a communication module, a memory, a processor, and a computer program stored on the memory and executable on the processor. The roadside sensor includes a lidar, a millimeter-wave radar, and a camera. The processor is respectively connected to the processor, the roadside sensor, and the communication module. When the processor executes the computer program, the following steps are implemented:

[0183] Obtain the deviation between the installation position of the camera on the first vehicle model and the installation position of the camera on the second vehicle model; the first vehicle model and the second vehicle model are different;

[0184] Obtain a first extrinsic matrix; the first extrinsic matrix is the extrinsic matrix of the camera on the first vehicle model;

[0185] Re-correct the first extrinsic matrix according to the deviation between the installation position of the camera on the first vehicle model and the installation position of the camera on the second vehicle model to obtain a second extrinsic matrix;

[0186] Obtain first image data; the first image data is the image data captured by the camera on the first vehicle model;

[0187] Perform a coordinate transformation on the pixel positions in the first image data according to the first extrinsic matrix and the second extrinsic matrix to obtain second image data.

[0188] In one embodiment, a vehicle is further provided, including a roadside sensor, a communication module, a memory, a processor, and a computer program stored on the memory and executable on the processor. The roadside sensor includes a lidar, a millimeter-wave radar, and a camera. The processor is respectively connected to the processor, the roadside sensor, and the communication module. When the processor executes the computer program, the steps in the above method embodiments are implemented.

[0189] In one embodiment, a computer device is provided, including a memory and a processor. A computer program is stored in the memory. When the processor executes the computer program, the following steps are implemented:

[0190] Obtain the deviation between the installation position of the camera on the first vehicle model and the installation position of the camera on the second vehicle model; the first vehicle model and the second vehicle model are different;

[0191] Obtain the first extrinsic matrix; the first extrinsic matrix is the extrinsic matrix of the camera on the first vehicle model;

[0192] According to the deviation between the installation position of the camera on the first vehicle model and the installation position of the camera on the second vehicle model, re-correct the first extrinsic matrix to obtain the second extrinsic matrix;

[0193] Obtain the first image data; the first image data is the image data captured by the camera on the first vehicle model;

[0194] According to the first extrinsic matrix and the second extrinsic matrix, perform coordinate transformation on the pixel positions in the first image data to obtain the second image data.

[0195] In one embodiment, a computer device is further provided, including a memory and a processor. A computer program is stored in the memory. When the processor executes the computer program, the steps in the above method embodiments are implemented.

[0196] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in this application are all information and data that have been authorized by the user or fully authorized by all parties.

[0197] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, database, or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memories. Non-volatile memories can include read-only memory (ROM), magnetic tapes, floppy disks, flash memories, optical memories, high-density embedded non-volatile memories, resistive random access memories (ReRAM), magnetoresistive random access memories (MRAM), ferroelectric random access memories (FRAM), phase change memories (PCM), graphene memories, etc. Volatile memories can include random access memory (RAM) or external cache memories, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The databases involved in the embodiments provided in the present application can include at least one of relational databases and non-relational databases. Non-relational databases can include distributed databases based on blockchain, etc., without limitation. The processors involved in the embodiments provided in the present application can be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, data processing logics based on quantum computing, etc., without limitation.

[0198] The technical features of the above embodiments can be combined arbitrarily. For the sake of concise description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope recorded in this specification.

[0199] The above embodiments only represent several implementation manners of the present application. The description is relatively specific and detailed, but it should not be construed as a limitation on the patent scope of the present application. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. A method for reusing road data, characterized in that, Applied to an assisted driving system, the system includes a camera, and the method includes: Obtain the deviation between the installation position of the camera on the first vehicle model and the installation position of the camera on the second vehicle model; the first vehicle model is different from the second vehicle model; Obtain the first extrinsic matrix; the first extrinsic matrix is the extrinsic matrix of the camera on the first vehicle model; According to the deviation between the installation position of the camera on the first vehicle model and the installation position of the camera on the second vehicle model, re-correct the first extrinsic matrix to obtain a second extrinsic matrix; the step of according to the deviation between the installation position of the camera on the first vehicle model and the installation position of the camera on the second vehicle model, re-correct the first extrinsic matrix to obtain a second extrinsic matrix includes: obtaining the installation position of the camera on the first vehicle model; determining the second mapping relationship between the installation position of the camera on the first vehicle model and the first extrinsic matrix; according to the deviation between the installation position of the camera on the first vehicle model and the installation position of the camera on the second vehicle model, and the second mapping relationship, re-correct the first extrinsic matrix to obtain a second extrinsic matrix; Obtain first image data; the first image data is the image data captured by the camera on the first vehicle model; According to the first extrinsic matrix and the second extrinsic matrix, perform coordinate transformation on the pixel positions in the first image data to obtain second image data; determine whether the communication matrix of the first vehicle model is the same as the communication matrix of the second vehicle model; if the communication matrix of the first vehicle model is not the same as the communication matrix of the second vehicle model, determine the first mapping relationship; the first mapping relationship represents the conversion relationship between the message signals in the communication matrix of the first vehicle model and the message signals in the communication matrix of the second vehicle model; according to the first mapping relationship, convert the communication matrix of the first vehicle model into the communication matrix of the second vehicle model.

2. The method according to claim 1, wherein After obtaining the second image data, the following steps are further included: Obtain the body data information corresponding to the first vehicle model; the body data information includes vehicle speed and direction; If the communication matrix of the first vehicle model is the same as the communication matrix of the second vehicle model, backfeed the second image data and the body data information into the software system corresponding to the second vehicle model.

3. The method according to claim 1, characterized in that, The step of performing coordinate transformation on the pixel positions in the first image data according to the first extrinsic matrix and the second extrinsic matrix to obtain second image data includes: According to the first extrinsic matrix, determine the first position coordinates of the imaging points of each corner point on the calibration board in the image; According to the second extrinsic matrix, determine the second position coordinates of the imaging points of each corner point on the calibration board in the image; According to the first position coordinates and the second position coordinates, determine the coordinate transformation relationship; According to the coordinate transformation relationship, perform coordinate transformation on the pixel positions in the first image data to obtain second image data.

4. A device for reusing road data, characterized in that, Applied to an assisted driving system, the system includes a camera, and the device includes: A first acquisition module, configured to acquire a deviation between the installation position of a camera on a first vehicle model and the installation position of a camera on a second vehicle model; the first vehicle model is different from the second vehicle model; A second acquisition module, configured to acquire a first extrinsic matrix; the first extrinsic matrix is the extrinsic matrix of the camera on the first vehicle model; An extrinsic matrix correction module, configured to re-correct the first extrinsic matrix according to the deviation between the installation position of the camera on the first vehicle model and the installation position of the camera on the second vehicle model, to obtain a second extrinsic matrix; the re-correcting the first extrinsic matrix according to the deviation between the installation position of the camera on the first vehicle model and the installation position of the camera on the second vehicle model to obtain a second extrinsic matrix includes: acquiring the installation position of the camera on the first vehicle model; determining a second mapping relationship between the installation position of the camera on the first vehicle model and the first extrinsic matrix; and re-correcting the first extrinsic matrix according to the deviation between the installation position of the camera on the first vehicle model and the installation position of the camera on the second vehicle model and the second mapping relationship, to obtain a second extrinsic matrix; A third acquisition module, configured to acquire first image data; the first image data is the image data captured by the camera on the first vehicle model; A coordinate transformation module, configured to perform coordinate transformation on the pixel positions in the first image data according to the first extrinsic matrix and the second extrinsic matrix, to obtain second image data; determining whether the communication matrix of the first vehicle model is the same as the communication matrix of the second vehicle model; if the communication matrix of the first vehicle model is different from the communication matrix of the second vehicle model, determining a first mapping relationship; the first mapping relationship represents the conversion relationship between the message signals in the communication matrix of the first vehicle model and the message signals in the communication matrix of the second vehicle model; and converting the communication matrix of the first vehicle model into the communication matrix of the second vehicle model according to the first mapping relationship.

5. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, the steps of the method according to any one of claims 1 to 3 are implemented.

6. A vehicle, comprising a roadside sensor, a communication module, a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the roadside sensor includes a lidar, a millimeter-wave radar, and a camera, and the processor is respectively connected to the processor, the roadside sensor, and the communication module, characterized in that, When the processor executes the computer program, the method for reusing road data according to any one of claims 1 to 3 is implemented.

Images