Offset calculation method and device for visual perception equipment of vehicle

Abstract

The invention provides an offset calculation method and device for visual perception equipment of a vehicle, and the method comprises the steps: firstly obtaining a plurality of images collected by the visual perception equipment of the vehicle in a first preset time period, and then determining the coordinates of a set point on the vertical central axis of each image in a two-dimensional coordinate system, the method comprises the following steps of: acquiring a set point of a vehicle, converting the set point into a coordinate under a vehicle body coordinate system through a preset perspective transformation model, determining coordinates of a plurality of specific points on a straight line where the current vehicle orientation is located under the vehicle body coordinate system, calculating deviation values of the set point and the corresponding specific point coordinates in each image, and finally fitting the deviation values of all the images to obtain a target image. And obtaining the shift amount of the vehicle visual perception equipment required to move relative to the current position. By adopting the technical scheme provided by the invention, the installation offset of the visual equipment does not need to be manually set, convenience and reliability are achieved, and the accuracy degree of offset setting can be effectively improved.

Description

Technical Field

[0001] The present application relates to the field of environmental perception technology, and in particular to a method and apparatus for calculating an offset of a visual perception device of a vehicle. Background Art

[0002] With the breakthrough of science and technology, agricultural production is moving towards intelligence and precision. The perception technology of agricultural machinery has been widely used in many aspects such as crop sowing, spraying, fertilization and harvesting operations. Among them, the use of machine vision and deep learning to realize the visual perception of agricultural machinery has gradually become one of the development trends. In actual production, the deviation of the installation of visual perception equipment will affect the detection results, so it is very necessary to correct the installation deviation of visual perception equipment. The core of the offset setting of visual perception equipment is to determine the offset between the vertical center axis of the image and the straight line where the actual driving direction is located. The current setting method is to transmit the perceived image back to the display controller in real time. The operator moves the vertical center axis of the current image through the interactive buttons to make it the same as the forward direction of the agricultural machinery. The resulting offset is the set offset.

[0003] However, in the actual agricultural machinery operation process, due to the influence of subjective factors such as the judgment and improper operation of the agricultural machinery operators, the obtained offset may still have errors, resulting in non-standard settings or offset errors still existing after the settings are completed, which will have a certain impact on the accuracy of subsequent visual perception operations. Summary of the invention

[0004] In view of this, an embodiment of the present application provides a method and device for calculating the offset of a vehicle's visual perception device. The present application does not require manual setting of the installation offset of the visual device, has convenience and reliability, can effectively improve the accuracy of the offset setting, and promote the offset setting method to develop in the direction of standardization and automation, thereby significantly reducing the interference caused by human errors.

[0005] This application mainly includes the following aspects:

[0006] In a first aspect, an embodiment of the present application provides a method for calculating an offset of a visual perception device of a vehicle, the method comprising:

[0007] Acquire a plurality of images captured by a visual perception device of the vehicle within a first preset time period;

[0008] Based on the multiple images, determine the coordinates of a corresponding set point on the vertical central axis of each of the multiple images in a two-dimensional coordinate system;

[0009] Based on a preset perspective transformation model, the coordinates of the set point corresponding to the vertical central axis of the image in the two-dimensional coordinate system are converted into the coordinates of the set point corresponding to the vertical central axis of the image in the vehicle body coordinate system;

[0010] Determine the coordinates of each specific point among a plurality of specific points on a straight line where the current vehicle is heading in a vehicle body coordinate system;

[0011] Determine a deviation value between the coordinates of the corresponding set point on the vertical central axis of the image and the coordinates of the corresponding specific point based on the coordinates of each specific point and the coordinates of the corresponding set point on the vertical central axis of the image in the vehicle body coordinate system;

[0012] The coordinates of the corresponding set points on the vertical central axis of all images are fitted with the deviation values ​​of the coordinates of the corresponding specific points to obtain the offset required for the vehicle's visual perception device to move relative to the current position.

[0013] Preferably, the preset perspective transformation model is constructed by the following steps:

[0014] Acquire a plurality of images captured by the visual perception device within a second preset time period;

[0015] Determine the coordinates of the corresponding calibration point on the vertical central axis of each image in the multiple images in a two-dimensional coordinate system;

[0016] A preset perspective transformation model is constructed based on the correspondence between the coordinates of the corresponding calibration points on the vertical central axis of each image in the multiple images in the two-dimensional coordinate system and the coordinates in the three-dimensional coordinate system.

[0017] Preferably, determining the deviation value between the coordinates of the corresponding set point on the vertical central axis of the image and the coordinates of the corresponding specific point based on the coordinates of each specific point and the coordinates of the corresponding set point on the vertical central axis of the image in the vehicle body coordinate system comprises:

[0018] Based on the coordinates of each specific point, determine the horizontal coordinate of the specific point having the same vertical coordinate as the set point corresponding to the vertical center axis of the image in the vehicle body coordinate system;

[0019] The difference between the horizontal coordinate of the set point corresponding to the vertical central axis of the image in the vehicle body coordinate system and the horizontal coordinate of the specific point with the same vertical coordinate is determined as the deviation value between the coordinate of the set point corresponding to the vertical central axis of the image and the coordinate of the corresponding specific point.

[0020] Preferably, the step of fitting the coordinates of the corresponding set points on the vertical central axis of all images with the deviation values ​​of the coordinates of the corresponding specific points comprises:

[0021] The Fourier function is approximately fitted to the deviation values ​​of the coordinates of the corresponding set points on the vertical central axis of all images and the coordinates of the corresponding specific points.

[0022] In a second aspect, an embodiment of the present application further provides an offset calculation device, the offset calculation device comprising:

[0023] An image acquisition module, which acquires a plurality of images collected by a visual perception device of the vehicle within a first preset time period;

[0024] A two-dimensional coordinate determination module, based on the multiple images, determines the coordinates of a corresponding set point on the vertical central axis of each of the multiple images in the two-dimensional coordinate system;

[0025] A coordinate conversion module, based on a preset perspective transformation model, converts the coordinates of the set point corresponding to the vertical central axis of the image in the two-dimensional coordinate system into the coordinates of the set point corresponding to the vertical central axis of the image in the vehicle body coordinate system;

[0026] A specific point coordinate determination module determines the coordinates of each specific point among a plurality of specific points on a straight line where the current vehicle is heading in a vehicle body coordinate system;

[0027] a deviation value calculation module, which determines the deviation value between the coordinates of the corresponding set point on the vertical central axis of the image and the coordinates of the corresponding specific point based on the coordinates of each specific point and the coordinates of the corresponding set point on the vertical central axis of the image in the vehicle body coordinate system;

[0028] The offset fitting module fits the coordinates of the corresponding set points on the vertical central axis of all images with the deviation values ​​of the coordinates of the corresponding specific points to obtain the offset that the vehicle's visual perception device needs to move relative to the current position.

[0029] Preferably, the coordinate transformation module constructs a preset perspective transformation model through the following steps:

[0030] Acquire a plurality of images captured by the visual perception device within a second preset time period;

[0031] Determine the coordinates of the corresponding calibration point on the vertical central axis of each image in the multiple images in a two-dimensional coordinate system;

[0032] A preset perspective transformation model is constructed based on the correspondence between the coordinates of the corresponding calibration points on the vertical central axis of each image in the multiple images in the two-dimensional coordinate system and the coordinates in the three-dimensional coordinate system.

[0033] Preferably, the deviation value calculation module is specifically used for:

[0034] Based on the coordinates of each specific point, determine the horizontal coordinate of the specific point having the same vertical coordinate as the set point corresponding to the vertical center axis of the image in the vehicle body coordinate system;

[0035] The difference between the horizontal coordinate of the set point corresponding to the vertical central axis of the image in the vehicle body coordinate system and the horizontal coordinate of the specific point with the same vertical coordinate is determined as the deviation value between the coordinate of the set point corresponding to the vertical central axis of the image and the coordinate of the corresponding specific point.

[0036] Preferably, when the offset fitting module is used to fit the coordinates of the corresponding set points on the vertical central axis of all images with the deviation values ​​of the coordinates of the corresponding specific points, it is also specifically used to:

[0037] The Fourier function is approximately fitted to the deviation values ​​of the coordinates of the corresponding set points on the vertical central axis of all images and the coordinates of the corresponding specific points.

[0038] In a third aspect, an embodiment of the present application further provides an electronic device, comprising: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate through the bus, and the machine-readable instructions are executed by the processor to execute the steps of the offset calculation method described in the first aspect or any possible implementation manner of the first aspect.

[0039] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of offset calculation described in the above-mentioned first aspect or any possible implementation manner of the first aspect are performed.

[0040] The embodiment of the present application provides a method and device for calculating the offset of a visual perception device of a vehicle. First, multiple images collected by the visual perception device of the vehicle within a first preset time period are obtained, and then the coordinates of the set points on the vertical central axis of each image in the two-dimensional coordinate system are determined, and then converted into coordinates in the vehicle body coordinate system through a preset perspective transformation model, and then the coordinates of multiple specific points on the straight line where the current vehicle is facing in the vehicle body coordinate system are determined, and then the deviation value between the set point in each image and the corresponding specific point coordinates is calculated, and finally the deviation values ​​of all images are fitted to obtain the offset required for the vehicle visual perception device to move relative to the current position. In this way, the present application does not need to manually set the installation offset of the visual device, has convenience and reliability, can effectively improve the accuracy of the offset setting, and promote the offset setting method to develop in the direction of standardization and automation, thereby significantly reducing the interference caused by human errors.

[0041] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0043] Figure 1 One of the flow charts of a method for calculating an offset of a visual perception device of a vehicle provided in an embodiment of the present application is shown;

[0044] Figure 2 An example diagram of the image perspective transformation process provided by an embodiment of the present application is shown;

[0045] Figure 3 A second flowchart of a method for calculating an offset of a visual perception device of a vehicle provided in an embodiment of the present application is shown;

[0046] Figure 4 A schematic diagram showing the structure of an offset calculation device of a visual perception device of a vehicle provided in an embodiment of the present application is shown;

[0047] Figure 5 A schematic structural diagram of an electronic device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0048] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It should be understood that the drawings in the present application only serve the purpose of explanation and description and are not used to limit the scope of protection of the present application. In addition, it should be understood that the schematic drawings are not drawn in real proportion. The flowchart used in this application shows the operations implemented according to some embodiments of the present application. It should be understood that the operations of the flowchart can be implemented out of sequence, and the steps without logical context can be reversed in order or implemented simultaneously. In addition, those skilled in the art, under the guidance of the content of the present application, can add one or more other operations to the flowchart, or remove one or more operations from the flowchart.

[0049] In addition, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0050] The following methods, devices, electronic devices or computer-readable storage media of the embodiments of the present application can be applied to any scenario that requires device offset calculation. The embodiments of the present application are not limited to specific application scenarios. Any method and device for calculating the offset of the visual perception device of the vehicle provided by the embodiments of the present application are within the scope of protection of the present application.

[0051] It is worth noting that with the breakthrough of science and technology, agricultural production is moving towards intelligence and precision. The perception technology of agricultural machinery has been widely used in many aspects such as crop sowing, spraying, fertilization and harvesting operations. Among them, the use of machine vision and deep learning to realize the visual perception of agricultural machinery has gradually become one of the development trends. In actual production, the deviation of the installation of visual perception equipment will affect the detection results, so it is very necessary to correct the deviation of the installation of visual perception equipment. The core of the offset setting of visual perception equipment is to determine the offset between the vertical central axis of the image and the straight line where the actual driving direction is located. The current setting method is to transmit the perceived image back to the display controller in real time. The operator moves the vertical central axis of the current image through the interactive buttons to make it the same as the forward direction of the agricultural machinery. The obtained offset is the set offset. However, in the actual agricultural machinery operation process, due to the influence of subjective factors such as the judgment and improper operation of the agricultural machinery operator, the obtained offset may still have errors, resulting in non-standard settings or offset errors after the setting is completed, which has a certain impact on the accuracy of subsequent visual perception operations.

[0052] In response to the above problems, the embodiments of the present application propose a method and device for calculating the offset of a vehicle's visual perception device. The present application does not require manual setting of the installation offset of the visual device, has the advantages of convenience and reliability, can effectively improve the accuracy of the offset setting, and promote the offset setting method to develop in the direction of standardization and automation, thereby significantly reducing the interference caused by human errors.

[0053] To facilitate the understanding of the present application, the technical solution provided by the present application is described in detail below in conjunction with specific embodiments.

[0054] See also Figure 1 , Figure 1One of the flowcharts of a method for calculating an offset of a visual perception device of a vehicle provided in an embodiment of the present application.

[0055] like Figure 1 As shown in , the offset calculation method of the visual perception device of the vehicle provided in the embodiment of the present application includes the following steps:

[0056] Step S101, acquiring a plurality of images captured by a visual perception device of a vehicle within a first preset time period.

[0057] Here, the visual perception device may include but is not limited to: a front view camera, a rear view camera, and a surround view camera. The first preset time period is set according to factors such as the fixed acquisition frequency, application scenario, and task requirements of the visual perception device itself. As an example, assuming that the first preset time period is 2 seconds and the visual perception device acquires 60 frames of images per second, the visual perception device of the vehicle acquires 120 images within the first preset time period.

[0058] Step S102: Based on the multiple images, determine the coordinates of a corresponding set point on the vertical central axis of each of the multiple images in a two-dimensional coordinate system.

[0059] Here, the selection of set points is determined based on specific task requirements. As an example, the set point selection rule is to select from the vertical center axis of the image according to the preset pixel step size. Assuming that the preset pixel step size is set to 10 pixels, for the first image captured by the vehicle's visual perception device, on its vertical center axis, a point 30 pixels away from the top of the image is selected as the set point; for the second image, a point 40 pixels away from the top of the image is selected on the vertical center axis as the set point. According to this rule, with a step size of 10 pixels, the corresponding set points can be determined in sequence on the vertical center axis of the subsequently captured images. During the driving process of the vehicle, due to various factors such as road conditions and vehicle driving status, the vehicle may shake. Due to the shaking of the vehicle, the horizontal coordinates of the corresponding set points on the vertical center axis of each image are inconsistent.

[0060] Step S103 , based on a preset perspective transformation model, the coordinates of the corresponding set point on the vertical central axis of the image in the two-dimensional coordinate system are converted into the coordinates of the corresponding set point on the vertical central axis of the image in the vehicle body coordinate system.

[0061] Here, the vehicle body coordinate system is a coordinate system with the vehicle itself as the origin. As an example, the preset perspective transformation model is constructed by the following steps:

[0062] First, a plurality of images captured by the visual perception device within a second preset time period are acquired.

[0063] Then, the coordinates of the corresponding calibration points on the vertical central axis of each image in the multiple images in the two-dimensional coordinate system are determined.

[0064] Finally, based on the correspondence between the coordinates of the corresponding calibration points on the vertical central axis of each of the multiple images in the two-dimensional coordinate system and the coordinates in the three-dimensional coordinate system, a preset perspective transformation model is constructed. Here, first, based on the correspondence between the coordinates of the corresponding calibration points on the vertical central axis of each of the multiple images in the two-dimensional coordinate system and the coordinates in the three-dimensional coordinate system, a perspective transformation matrix is ​​determined. As an example, the perspective transformation matrix can be determined by the OpenCV perspective transformation four-point calibration method, and the perspective transformation matrix M can be expressed by formula (1).

[0065]

[0066] Among them, a ij (i=1,2,3;j=1,2,3) are the coefficients of the preset perspective transformation matrix, a ij The coordinates of the calibration points corresponding to the vertical central axis of each image in the multiple images in the two-dimensional coordinate system and the coordinates of the calibration points in the three-dimensional coordinate system are obtained. As an example, the corresponding relationship between the coordinates of the calibration points corresponding to the vertical central axis of each image in the multiple images in the two-dimensional coordinate system and the coordinates of the calibration points in the three-dimensional coordinate system can be expressed by formula (2).

[0067]

[0068] Where (x, y,) is the coordinate of the corresponding calibration point on the vertical central axis of the image in the two-dimensional coordinate system, and (x', y', z') is the coordinate of the corresponding calibration point on the vertical central axis of the image in the three-dimensional coordinate system, where z' is perpendicular to the ground. It can be seen from formula 2 that the coordinate of the corresponding calibration point on the vertical central axis of each image in the multiple images in the three-dimensional coordinate system can be expressed by formula (3).

[0069]

[0070] By using formula (3), after the coordinates of the corresponding calibration points on the vertical central axis of each image in the multiple images in the two-dimensional coordinate system are converted into coordinates in the three-dimensional coordinate system, the perspective transformation can keep the straight lines in the original image as straight lines after the perspective transformation. In order to achieve the perspective effect, x', y', and z' in formula (3) are divided by z' to obtain the preset perspective transformation model. As an example, the preset perspective transformation model can be expressed by formula (4).

[0071]

[0072] Among them, (x”, y”) is the coordinate of the set point corresponding to the vertical central axis of the image in the vehicle body coordinate system. Formula (4) can be used to convert the coordinate of the set point corresponding to the vertical central axis of the image in the two-dimensional coordinate system into the coordinate of the set point corresponding to the vertical central axis of the image in the vehicle body coordinate system. After the conversion, it is ensured that the coordinate of the point on the vertical central axis of the image and the coordinate of the point on the straight line where the vehicle is facing are in the same coordinate system. As an example, the effect of converting the coordinate of the set point corresponding to the vertical central axis of the image in the two-dimensional coordinate system into the coordinate of the set point corresponding to the vertical central axis of the image in the vehicle body coordinate system is as follows: Figure 2 as shown in .

[0073] Step S104, determining the coordinates of each specific point among a plurality of specific points on a straight line where the current vehicle is facing in the vehicle body coordinate system.

[0074] Here, the selection rule of the specific point is that the point on the straight line where the current vehicle is heading in the vehicle body coordinate system is the same as the point on the vertical center axis of each image whose vertical coordinate in the vehicle body coordinate system is the same.

[0075] Step S105, based on the coordinates of each specific point and the coordinates of the corresponding set point on the vertical central axis of the image in the vehicle body coordinate system, determine the deviation value between the coordinates of the corresponding set point on the vertical central axis of the image and the coordinates of the corresponding specific point.

[0076] Combine the following Figure 3 To illustrate how to determine the deviation value between the coordinates of the corresponding set point on the vertical central axis of the image and the coordinates of the corresponding specific point based on the coordinates of each specific point and the coordinates of the corresponding set point on the vertical central axis of the image in the vehicle body coordinate system.

[0077] See also Figure 3 , Figure 3 The present invention is a flowchart of the second method for calculating the offset of a visual perception device of a vehicle provided in an embodiment of the present application.

[0078] like Figure 3 As shown in FIG. 1 , regarding step S105, in a specific implementation, as an example, the following steps may be included:

[0079] Step S1051, based on the coordinates of each specific point, determine the horizontal coordinate of the specific point having the same vertical coordinate as the set point corresponding to the vertical center axis of the image in the vehicle body coordinate system.

[0080] Step S1052, determining the difference between the horizontal coordinate of the set point corresponding to the vertical central axis of the image in the vehicle body coordinate system and the horizontal coordinate of the specific point with the same vertical coordinate as the deviation value between the coordinate of the set point corresponding to the vertical central axis of the image and the coordinate of the corresponding specific point.

[0081] Step S106, fitting the coordinates of the corresponding set points on the vertical central axis of all images with the deviation values ​​of the coordinates of the corresponding specific points to obtain the offset required for the vehicle's visual perception device to move relative to the current position.

[0082] Here, as an example, the coordinates of the set points corresponding to the vertical central axis of all images and the deviation values ​​of the coordinates of the corresponding specific points are approximated by Fourier function fitting. Fourier function approximation fitting is a mathematical method widely used in the field of signal processing and data analysis. It can represent a complex function as a superposition of a series of simple sine and cosine functions. As an example, the expression after the coordinates of the set points corresponding to the vertical central axis of all images and the deviation values ​​of the coordinates of the corresponding specific points are approximated by Fourier function fitting can be expressed by formula (5).

[0083] r=a0+a1×cos(c+w)+b1×sin(c+w) (5)

[0084] Where r is the deviation between the coordinates of the corresponding set point on the vertical central axis of the image and the coordinates of the corresponding specific point, a0 is the offset required for the vehicle's visual perception device to move relative to the current position, a1 is the periodically changing cosine coefficient, b1 is the periodically changing sine coefficient, c is the serial number corresponding to the deviation between the coordinates of the corresponding set point on the vertical central axis of the image and the coordinates of the corresponding specific point, w is the phase offset, w=2π / T, where T is the period. As an example, if the deviation between the coordinates of the corresponding set point on the vertical central axis of the first image collected by the vehicle's visual perception device and the coordinates of the corresponding specific point is 1, then r is 1 and c is 1. If the deviation between the coordinates of the corresponding set point on the vertical central axis of the second image collected by the vehicle's visual perception device and the coordinates of the corresponding specific point is 3, then r is 3 and c is 2. As can be seen from formula (5), a0 in the expression after the Fourier function approximation fitting of the deviation between the coordinates of the corresponding set point on the vertical central axis of all images and the coordinates of the corresponding specific point is the offset required for the vehicle's visual perception device to move relative to the current position.

[0085] In this application, after calculating the offset required for the visual perception device of the vehicle to move relative to the current position by the above method, the offset is transmitted back to the controller of the agricultural machinery by communicating with the vehicle as a whole such as CAN signal or ISOBUS, and is also fed back to the offset setting system to complete the offset setting, thereby realizing adaptive offset calculation. On this basis, the image that has undergone perspective transformation is processed by inverse perspective transformation, so that the agricultural machinery operator can intuitively observe the image result after the offset setting is completed.

[0086] An embodiment of the present application provides a method for calculating the offset of a vehicle's visual perception device. Through the method, there is no need to manually set the installation offset of the visual device. The method is convenient and reliable, can effectively improve the accuracy of the offset setting, and promote the offset setting method to develop in a standardized and automated direction, thereby significantly reducing the interference caused by human errors.

[0087] Based on the same application concept, the embodiments of the present application also provide an offset calculation device for the vehicle's visual perception device corresponding to the offset calculation method for the vehicle's visual perception device provided in the above embodiments. Since the principle of solving the problem by the device in the embodiments of the present application is similar to the offset calculation method for the vehicle's visual perception device in the above embodiments of the present application, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0088] See also Figure 4 , Figure 4 A schematic diagram of the structure of an offset calculation device for a visual perception device of a vehicle provided in an embodiment of the present application.

[0089] like Figure 4 As shown in FIG. 4 , the offset calculation device 410 of the visual perception device of the vehicle provided in the embodiment of the present application includes:

[0090] The image acquisition module 411 acquires a plurality of images collected by the visual perception device of the vehicle within a first preset time period;

[0091] A two-dimensional coordinate determination module 412 determines, based on the multiple images, the coordinates of a corresponding set point on the vertical central axis of each of the multiple images in the two-dimensional coordinate system;

[0092] A coordinate conversion module 413 converts the coordinates of the set point corresponding to the vertical central axis of the image in the two-dimensional coordinate system into the coordinates of the set point corresponding to the vertical central axis of the image in the vehicle body coordinate system based on a preset perspective transformation model;

[0093] A specific point coordinate determination module 414 determines the coordinates of each specific point among a plurality of specific points on a straight line where the current vehicle is heading in the vehicle body coordinate system;

[0094] The deviation value calculation module 415 determines the deviation value between the coordinates of the corresponding set point on the vertical central axis of the image and the coordinates of the corresponding specific point based on the coordinates of each specific point and the coordinates of the corresponding set point on the vertical central axis of the image in the vehicle body coordinate system;

[0095] The offset fitting module 416 fits the coordinates of the corresponding set points on the vertical central axis of all images with the deviation values ​​of the coordinates of the corresponding specific points to obtain the offset required for the vehicle's visual perception device to move relative to the current position.

[0096] Preferably, the coordinate transformation module 413 constructs a preset perspective transformation model through the following steps:

[0097] Acquire a plurality of images captured by the visual perception device within a second preset time period;

[0098] Determine the coordinates of the corresponding calibration point on the vertical central axis of each image in the multiple images in a two-dimensional coordinate system;

[0099] A preset perspective transformation model is constructed based on the correspondence between the coordinates of the corresponding calibration points on the vertical central axis of each image in the multiple images in the two-dimensional coordinate system and the coordinates in the three-dimensional coordinate system.

[0100] Preferably, the deviation value calculation module 414 is specifically used for:

[0101] Based on the coordinates of each specific point, determine the horizontal coordinate of the specific point having the same vertical coordinate as the set point corresponding to the vertical center axis of the image in the vehicle body coordinate system;

[0102] The difference between the horizontal coordinate of the set point corresponding to the vertical central axis of the image in the vehicle body coordinate system and the horizontal coordinate of the specific point with the same vertical coordinate is determined as the deviation value between the coordinate of the set point corresponding to the vertical central axis of the image and the coordinate of the corresponding specific point.

[0103] Preferably, when the offset fitting module 416 is used to fit the coordinates of the corresponding set points on the vertical central axis of all images with the deviation values ​​of the coordinates of the corresponding specific points, it is also specifically used to:

[0104] The Fourier function is approximately fitted to the deviation values ​​of the coordinates of the corresponding set points on the vertical central axis of all images and the coordinates of the corresponding specific points.

[0105] An embodiment of the present application provides an offset calculation device for a vehicle's visual perception device. Through the device, there is no need to manually set the installation offset of the visual device. The device is convenient and reliable, can effectively improve the accuracy of the offset setting, and promote the offset setting method to develop in a standardized and automated direction, thereby significantly reducing the interference caused by human errors.

[0106] See also Figure 5 , Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0107] like Figure 5 As shown in , the electronic device 500 includes a processor 510 , a memory 520 and a bus 530 .

[0108] The memory 520 stores machine-readable instructions executable by the processor 510. When the electronic device 500 is running, the processor 510 communicates with the memory 520 via the bus 530. When the machine-readable instructions are executed by the processor 510, the above-mentioned Figure 1 and Figure 3 The steps of the method for calculating the offset of the visual perception device of the vehicle in the method embodiment shown, the specific implementation method can be found in the method embodiment, and will not be repeated here.

[0109] The present application also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the computer program can execute the above-mentioned Figure 1 and Figure 3 The steps of the method for calculating the offset of the visual perception device of the vehicle in the method embodiment shown, the specific implementation method can be found in the method embodiment, and will not be repeated here.

[0110] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, the specific working process of the system and device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here. In the several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0111] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0112] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0113] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc., various media that can store program codes.

[0114] The above are only specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for calculating the offset of a visual perception device of a vehicle, characterized in that: The offset calculation method comprises: Acquire a plurality of images captured by a visual perception device of the vehicle within a first preset time period; Based on the multiple images, determine the coordinates of a corresponding set point on the vertical central axis of each of the multiple images in a two-dimensional coordinate system; Based on a preset perspective transformation model, the coordinates of the set point corresponding to the vertical central axis of the image in the two-dimensional coordinate system are converted into the coordinates of the set point corresponding to the vertical central axis of the image in the vehicle body coordinate system; Determine the coordinates of each specific point among a plurality of specific points on a straight line where the current vehicle is heading in a vehicle body coordinate system; Determine a deviation value between the coordinates of the corresponding set point on the vertical central axis of the image and the coordinates of the corresponding specific point based on the coordinates of each specific point and the coordinates of the corresponding set point on the vertical central axis of the image in the vehicle body coordinate system; The coordinates of the corresponding set points on the vertical central axis of all images are fitted with the deviation values ​​of the coordinates of the corresponding specific points to obtain the offset required for the vehicle's visual perception device to move relative to the current position.

2. The offset calculation method according to claim 1, characterized in that: Construct a preset perspective transformation model by following these steps: Acquire a plurality of images captured by the visual perception device within a second preset time period; Determine the coordinates of the corresponding calibration point on the vertical central axis of each image in the multiple images in a two-dimensional coordinate system; A preset perspective transformation model is constructed based on the correspondence between the coordinates of the corresponding calibration points on the vertical central axis of each image in the multiple images in the two-dimensional coordinate system and the coordinates in the three-dimensional coordinate system.

3. The offset calculation method according to claim 1, characterized in that: The step of determining the deviation between the coordinates of the corresponding set point on the vertical central axis of the image and the coordinates of the corresponding specific point based on the coordinates of each specific point and the coordinates of the corresponding set point on the vertical central axis of the image in the vehicle body coordinate system comprises: Based on the coordinates of each specific point, determine the horizontal coordinate of the specific point having the same vertical coordinate as the set point corresponding to the vertical center axis of the image in the vehicle body coordinate system; The difference between the horizontal coordinate of the set point corresponding to the vertical central axis of the image in the vehicle body coordinate system and the horizontal coordinate of the specific point with the same vertical coordinate is determined as the deviation value between the coordinate of the set point corresponding to the vertical central axis of the image and the coordinate of the corresponding specific point.

4. The offset calculation method according to claim 1, characterized in that: The step of fitting the coordinates of the set points corresponding to the vertical central axis of all images and the deviation values ​​of the coordinates of the corresponding specific points comprises: The Fourier function is approximately fitted to the deviation values ​​of the coordinates of the corresponding set points on the vertical central axis of all images and the coordinates of the corresponding specific points.

5. An offset calculation device for a visual perception device of a vehicle, characterized in that: The offset calculation device comprises: An image acquisition module, which acquires a plurality of images collected by a visual perception device of the vehicle within a first preset time period; A two-dimensional coordinate determination module, based on the multiple images, determines the coordinates of a corresponding set point on the vertical central axis of each of the multiple images in the two-dimensional coordinate system; A coordinate conversion module, based on a preset perspective transformation model, converts the coordinates of the set point corresponding to the vertical central axis of the image in the two-dimensional coordinate system into the coordinates of the set point corresponding to the vertical central axis of the image in the vehicle body coordinate system; A specific point coordinate determination module determines the coordinates of each specific point among a plurality of specific points on a straight line where the current vehicle is heading in a vehicle body coordinate system; a deviation value calculation module, which determines the deviation value between the coordinates of the corresponding set point on the vertical central axis of the image and the coordinates of the corresponding specific point based on the coordinates of each specific point and the coordinates of the corresponding set point on the vertical central axis of the image in the vehicle body coordinate system; The offset fitting module fits the coordinates of the corresponding set points on the vertical central axis of all images with the deviation values ​​of the coordinates of the corresponding specific points to obtain the offset that the vehicle's visual perception device needs to move relative to the current position.

6. The offset calculation device according to claim 5, characterized in that: The coordinate transformation module constructs a preset perspective transformation model through the following steps: Acquire a plurality of images captured by the visual perception device within a second preset time period; Determine the coordinates of the corresponding calibration point on the vertical central axis of each image in the multiple images in a two-dimensional coordinate system; A preset perspective transformation model is constructed based on the correspondence between the coordinates of the corresponding calibration points on the vertical central axis of each image in the multiple images in the two-dimensional coordinate system and the coordinates in the three-dimensional coordinate system.

7. The offset calculation device according to claim 5, characterized in that: The deviation value calculation module is specifically used for: Based on the coordinates of each specific point, determine the horizontal coordinate of the specific point having the same vertical coordinate as the set point corresponding to the vertical center axis of the image in the vehicle body coordinate system; The difference between the horizontal coordinate of the set point corresponding to the vertical central axis of the image in the vehicle body coordinate system and the horizontal coordinate of the specific point with the same vertical coordinate is determined as the deviation value between the coordinate of the set point corresponding to the vertical central axis of the image and the coordinate of the corresponding specific point.

8. The offset calculation device according to claim 5, characterized in that: When the offset fitting module is used to fit the coordinates of the corresponding set points on the vertical central axis of all images with the deviation values ​​of the coordinates of the corresponding specific points, it is also specifically used to: The Fourier function is approximately fitted to the deviation values ​​of the coordinates of the corresponding set points on the vertical central axis of all images and the coordinates of the corresponding specific points.

9. An electronic device, characterized in that: include: A processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate through the bus, and the machine-readable instructions are executed by the processor to execute the steps of the offset calculation method as described in any one of claims 1 to 4.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the offset calculation method according to any one of claims 1 to 4 are executed.

Images