Vehicle image processing system, method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a vehicle image processing system, method and device, electronic equipment and a storage medium. The system comprises an upper computer and a model car, the model car is provided with a controller and a fixing structure for deploying a plurality of first visual sensors, and the fixing structure allows the first visual sensors to adjust positions and shooting angles; the upper computer is connected with the controller and is used for sending a first control instruction to the controller; the upper computer is also used for acquiring the first image acquired by the first visual sensor and generating a look-around image corresponding to the model car according to the first image; the controller is used for driving the model car to move under the action of the first control instruction. The model car is used for replacing a car, and the fixing structure which enables the first visual sensor to adjust the position and the angle easily is arranged on the model car, so that the difficulty that the first visual sensor collects images around the model car can be reduced, the image acquisition efficiency is improved, and the AVM test cost is reduced.

Description

Technical Field

[0001] This application relates to the field of vehicle image processing, and more particularly to a vehicle image processing system, method, apparatus, electronic device, and storage medium. Background Technology

[0002] With the rapid development of the automotive industry, cars are becoming increasingly feature-rich. For example, many vehicles can use AVM (Around View Monitor) to display panoramic images of the area around the vehicle.

[0003] Currently, AVM systems need to acquire a large number of images from multiple directions around the vehicle for functional testing and optimization. This usually requires setting up cameras on real cars to collect images, which is costly and limited by the vehicle's structure, restricting camera deployment locations and making image acquisition difficult. Summary of the Invention

[0004] This application provides a vehicle image processing system, method, device, electronic device, and storage medium to reduce AVM testing costs and image acquisition difficulty.

[0005] In a first aspect, embodiments of this application provide a vehicle image processing system, including a host computer and a model vehicle;

[0006] The model vehicle is equipped with a controller and a fixed structure for deploying multiple first vision sensors, the fixed structure allowing the first vision sensors to adjust their position and shooting angle;

[0007] The host computer is connected to the controller and is used to send a first control command to the controller;

[0008] The host computer is also used to acquire the first image collected by the first vision sensor and generate a surround view image corresponding to the model vehicle based on the first image.

[0009] The controller is used to drive the model car to move under the action of the first control command.

[0010] In one possible implementation, the fixing structure includes a plurality of bases disposed on the side of the model vehicle, each base for placing at least one of the first vision sensors.

[0011] In one possible implementation, the fixing structure includes a clamp and a support tube or guide rail disposed on the side of the model vehicle, wherein the first vision sensor is fixed to the support tube or the guide rail by the clamp.

[0012] In one possible implementation, the host computer is also used to send a second control command to the first vision sensor, the second control command being used to control the first vision sensor to adjust its position and / or shooting angle on the model vehicle.

[0013] In one possible implementation, the vehicle monitoring image processing system further includes a display mounted on the model vehicle, the display being connected to the host computer, and the display being used to receive and display images transmitted by the host computer.

[0014] In one possible implementation, the vehicle image processing system further includes a second vision sensor connected to the host computer, the second vision sensor being used to acquire top-down images of the model vehicle.

[0015] Secondly, embodiments of this application provide a vehicle image processing method, which can be applied to a host computer in any of the first aspects described above. The method includes:

[0016] Acquire first images of the model vehicle in a preset direction, captured by multiple first image acquisition devices;

[0017] The first panoramic image is obtained by processing multiple first images using a preset panoramic algorithm.

[0018] In one possible implementation, the vehicle image processing method further includes:

[0019] Obtain a top-down image of the model vehicle;

[0020] The surround view algorithm is optimized based on the first surround view image and the top view image.

[0021] Thirdly, embodiments of this application provide a vehicle image processing apparatus, comprising:

[0022] The acquisition module is used to acquire first images of the model vehicle in a preset direction, acquired by multiple first image acquisition devices;

[0023] The image processing module is used to process multiple first images using a preset surround view algorithm to obtain a first surround view image.

[0024] Fourthly, embodiments of this application provide an electronic processing device, including: a memory and a processor;

[0025] The memory stores computer-executed instructions;

[0026] The processor executes computer execution instructions stored in the memory, causing the processor to perform the first aspect and / or various possible implementations of the first aspect as described above.

[0027] Fifthly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the first aspect and / or various possible implementations of the first aspect.

[0028] In a sixth aspect, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the first aspect and / or various possible implementations of the first aspect.

[0029] The vehicle image processing system, method, apparatus, electronic device, and storage medium provided in this application embodiment replace automobiles with model cars and set up fixed structures on the model cars that make it easy for the first vision sensor to adjust its position and angle. This can reduce the difficulty for the first vision sensor to acquire images around the model car, improve image acquisition efficiency, and reduce AVM testing costs. Attached Figure Description

[0030] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0031] Figure 1 A structural schematic diagram of a model car provided in this application;

[0032] Figure 2 A structural schematic diagram of another model car provided in this application;

[0033] Figure 3 This is a structural schematic diagram of a support tube and clamp provided in this application;

[0034] Figure 4 A structural schematic diagram of yet another model car provided in this application;

[0035] Figure 5 A schematic diagram illustrating the principle of acquiring top-view images provided in this application;

[0036] Figure 6 A flowchart illustrating a vehicle image processing method provided in this application;

[0037] Figure 7 A schematic diagram of the structure of a vehicle image processing device provided in this application;

[0038] Figure 8 This is a schematic diagram of the structure of an electronic device provided in this application.

[0039] Explanation of reference numerals in the attached figures:

[0040] 1-Controller; 2-First vision sensor; 3-Fixed structure; 4-Vehicle body; 5-Wheel; 6-Display; 7-Second vision sensor; Support tube-31; Fixture-32.

[0041] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0042] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0043] The terms "comprising," "including," or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, product, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, product, or apparatus. Without further limitation, the presence of other identical or equivalent elements in the process, method, product, or apparatus that includes said elements is not excluded. For example, the use of terms such as "first," "second," etc., is to denote names and does not indicate any particular order.

[0044] The terms “vertical,” “horizontal,” “left,” “right,” “up,” “down,” “front,” “back,” “circumferential,” “direction of travel,” and similar expressions used herein are based on the orientation or positional relationship shown in the accompanying drawings and are used only for the convenience of describing the invention and simplifying the description, and are not intended to indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the invention.

[0045] Unless otherwise defined, the technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the specification of this invention is for the purpose of describing particular embodiments only and is not intended to limit the invention. The terms “and / or,” “and / or,” and “at least one of” as used herein include any and all combinations of one or more of the associated listed items. It should be noted that the connections, links, etc., described in this disclosure can be direct connections via interfaces or pins between devices, connections via leads, or wireless connections (communication connections).

[0046] First, let me explain the terms used in this application:

[0047] AVM stands for Around View Monitor, which refers to providing a panoramic view of the vehicle's surroundings through multiple cameras.

[0048] AVM (Aerial View Monitor) is playing an increasingly important role in the automotive industry. By providing a 360-degree panoramic view, it helps drivers better understand the environment around the vehicle, improving driving safety and convenience. The principle of AVM is to stitch together images from different cameras on the vehicle to generate a bird's-eye view (top-down view). Testing and optimization of the AVM system can include adjusting the parameters of the image processing algorithm, as well as the camera positions and shooting angles. During testing, a sufficient number of images need to be collected, and the parameters of the image processing algorithm and camera positions are continuously optimized based on the collected images.

[0049] However, the inventors discovered through research that existing AVM testing methods mainly involve deploying cameras on real cars to collect images. Each test requires the use of a car, which is costly. Furthermore, the locations on a car where cameras can be deployed are limited, making it difficult to collect enough images for optimizing the AVM system.

[0050] The inventors then conceived of using a simpler model car instead of a real vehicle, incorporating a structure on the model car that facilitates camera position adjustment. This allows for flexible adjustment of the camera's horizontal and vertical position, as well as its shooting angle, via manual or motor-driven methods. This not only reduces the testing cost of the AVM system but also enhances the camera's flexibility and simplifies image acquisition.

[0051] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0052] According to one aspect of this application, a vehicle image processing system is provided. In one embodiment, the vehicle image processing system includes a host computer and a model vehicle.

[0053] Figure 1 A structural schematic diagram of a model car provided in this application. See below for reference. Figure 1 The structure and function of the model car will be further illustrated by example.

[0054] The model car is equipped with a controller 1 and a fixed structure 3 for deploying multiple first vision sensors 2. The fixed structure 3 allows the first vision sensors to adjust their position and shooting angle. The host computer is connected to the controller 1 and is used to send first control commands to the controller 1. The host computer is also used to acquire the first image collected by the first vision sensors 2 and generate a surround view image of the model car based on the first image. The controller 1 is used to drive the model car to move under the action of the first control command.

[0055] In some possible implementations, the model car may include a body 4 and wheels 5. The body 4 may include structures such as bearings for rotating the wheels 5, as well as structures for housing the controller 1. For example, the body 4 may include structures such as front and rear axles and a chassis. The body 4 may be a frame made of materials such as metal or plastic; this embodiment does not limit the materials used for the model car.

[0056] In some possible implementations, the model car may be equipped with a controller 1. The controller 1 may include a processor with data processing capabilities and a motor connected to the processor. The motor may be connected to the wheels 5 of the model car. The motor may receive instructions from the processor and drive the wheels 5 to steer and rotate under the action of the instructions. The processor may be a microcontroller.

[0057] In some possible implementations, the first visual sensor 2 may include various cameras, video cameras, or other devices with image acquisition capabilities. The model car body 4 includes a fixed structure 3 for deploying the first visual sensor 2. This fixed structure 3 may be a bracket set on the side of the body 4. The first visual sensor 2 can be placed directly on this bracket, or it can be fixed to the bracket by magnetic adsorption or mechanical connection. The sides of the body 4 may include vertical sides such as the front side, rear side, left side, and right side. For example, one first visual sensor 2 can be deployed on each of the four sides of the model car (front, rear, left, and right). The first visual sensor 2 deployed on the front side has its shooting direction facing forward of the model car.

[0058] In some possible implementations, the host computer can be a terminal device such as a smartphone or computer with data processing capabilities. The host computer can be connected to both the controller 1 and the first vision sensor 2; this connection can be electrical or wireless. The host computer can send commands to the controller 1 to drive the model vehicle. The host computer can also acquire images collected by the first vision sensor 2 and can stitch and fuse multiple images collected by the first vision sensor 2 into a surround-view image based on a surround-view algorithm.

[0059] In some possible implementations, controller 1 can be connected to first vision sensor 2, which can transmit the acquired images to controller 1, and controller 1 can then transmit the images to a host computer.

[0060] It should be noted that, Figure 1 This is merely an exemplary schematic diagram provided to illustrate the structure of the model car and does not represent all possible forms of representation that the model car may include. Figure 1 The image only shows the first visual sensor 2 deployed on the front side of the model car and the corresponding fixing structure 3. The first visual sensor 2 can be deployed on the other three sides of the model car.

[0061] In one embodiment, the fixing structure 3 includes multiple bases disposed on the side of the model vehicle, each base for placing at least one of the first vision sensors 2. The base can be a bracket or mounting plate for fixing the first vision sensor 2. The base may have mounting holes for fixing the first vision sensor 2; after aligning the first vision sensor 2 with the mounting holes on the base, the first vision sensor 2 can be fixed to the base using screws and nuts.

[0062] For example, multiple bases can be set on each side of the model car, with different positions on the bases. The first vision sensor 2 can be placed on any base, and the effect of the image captured by the base at that position can be evaluated based on the image captured by the first vision sensor 2. Thus, the position of the camera can be adjusted according to the image effect. For example, the front side of the model car has a first base and a second base. The first image captured by the first vision sensor 2 on the first base and the second base can be acquired respectively, and the effect of the surround view image can be determined based on the first image. The optimal deployment position of the first vision sensor 2 can then be determined based on the surround view image effect.

[0063] In the above embodiments, by using a model car instead of a car and setting a fixed structure 3 on the model car that makes it easy to adjust the position and angle of the first vision sensor 2, the difficulty of the first vision sensor 2 to collect images around the model car can be reduced, the image acquisition efficiency can be improved, and the AVM test cost can be reduced.

[0064] In one embodiment, the fixing structure 3 includes a clamp 32 and a support tube 31 or guide rail disposed on the side of the model car, wherein the first visual sensor 2 is fixed to the support tube 31 or the guide rail by the clamp 32.

[0065] like Figure 2 As shown, support tubes 31 can be installed on the four sides of the model car body. Figure 2 (Only a portion is shown in the image) serves as the fixed structure 3 for the first visual sensor 2. The first visual sensor 2 can be accessed via, for example... Figure 3 The clamp 32 shown is fixed to the support tube 31. Specifically, the clamp 32 may include a support platform for supporting the first visual sensor 2 and a column below the support platform. The column may have multiple through holes, each with a diameter larger than the diameter of the support tube 31. After the support tube 31 is inserted into the through hole, the clamp 32 can slide along the support tube 31, thereby adjusting the horizontal position of the first visual sensor 2. The height of the first visual sensor 2 can also be adjusted by inserting the support tube 31 into different through holes.

[0066] In some possible implementations, the support tube 31 can be replaced with a guide rail, and the first visual sensor 2 can be fixed on a slider of the guide rail. The position of the first visual sensor 2 can be adjusted by moving the slider. In some other possible implementations, a lifting mechanism can be used to adjust the height of the guide rail, thereby adjusting the height of the first visual sensor 2.

[0067] In one embodiment, the host computer is further configured to send a second control command to the first vision sensor 2, the second control command being configured to control the first vision sensor 2 to adjust its position and / or shooting angle on the model vehicle.

[0068] Specifically, the host computer can be electrically connected or wirelessly connected to the first vision sensor 2, and can send control commands to the first vision sensor 2. The first vision sensor 2 can perform operations such as moving its position and rotating the lens to shoot according to the commands.

[0069] In the above embodiments, by sending instructions from the host computer to the first visual sensor 2, the first visual sensor 2 can be controlled to automatically adjust its position and shooting angle without manual adjustment, which can improve the image acquisition efficiency of the first visual sensor 2.

[0070] In one embodiment, the vehicle monitoring image processing system further includes a display 6 mounted on the model vehicle, the display 6 being connected to the host computer, and the display 6 being used to receive and display images transmitted by the host computer.

[0071] Specifically, such as Figure 4As shown, the model vehicle can also be equipped with a display 6, which can be electrically or wirelessly connected to a host computer. The display 6 can receive images transmitted from the host computer and display the images on its screen. The images displayed on the display 6 may include the first image captured by the first vision sensor 2, or a surround view image generated by the host computer based on multiple first images.

[0072] In the above embodiments, by setting a display 6 on the model vehicle, the first image and surround view image of the model vehicle can be observed in real time, which facilitates the adjustment of the movement of the model vehicle and the position and angle of the first vision sensor 2, thereby improving the efficiency of AVM testing.

[0073] In one embodiment, the vehicle image processing system further includes a second vision sensor 7 connected to the host computer, the second vision sensor 7 being used to acquire top-down images of the model vehicle.

[0074] Specifically, such as Figure 5 As shown, a second visual sensor 7 can be set at a preset height above the model car. The second visual sensor 7 is kept at a certain distance from the top of the model car so that a top-down image containing the model car can be captured through the second visual sensor 7.

[0075] In the above embodiments, the top-down image of the model car can be obtained through the second vision sensor 7, and the top-down image can be compared with the surround-view image generated by the host computer based on the first image. The top-down image is used as the ground truth data to correct the generation method of the surround-view image, thereby improving the accuracy of the surround-view image.

[0076] In one embodiment, the vehicle monitoring image processing system may further include a power module, which may be connected to any one or more of the following devices: controller 1, first vision sensor 2, second vision sensor 7, and display 6, to supply power to these devices.

[0077] According to one aspect of this application, a vehicle image processing method is provided. For example... Figure 6 As shown, the method may include:

[0078] S601. Acquire first images of the model vehicle in a preset direction, acquired by multiple first image acquisition devices.

[0079] The preset direction can be the direction of the side of the vehicle body where the first image acquisition device is located relative to the entire model vehicle. For example, if the first image acquisition device is deployed on the front side of the model vehicle, the preset direction can be the front of the model vehicle. The first image acquisition device can be the first vision sensor 2 in the above embodiments.

[0080] Specifically, after setting the position and shooting angle of the first image acquisition device, the first image acquisition device can acquire a first image, and the data related to the first image can be transferred from the first image acquisition device to the host computer. For example, data can be read from the storage chip of the first image acquisition device and transferred to the host computer, or a communication connection can be established between the first image acquisition device and the host computer, through which the first image acquisition device transmits the data related to the first image to the host computer.

[0081] In some possible implementations, the model vehicle can be moved by a controller while the first image acquisition device is acquiring images of the area around the model vehicle, so that the first image acquisition device can acquire images of the model vehicle moving.

[0082] In some possible implementations, the position and shooting angle of the first image acquisition device can be adjusted manually or by sending instructions from a host computer to the first image acquisition device, and the first image acquired by the first image acquisition device can continue to be acquired after the adjustment is completed.

[0083] In some possible implementations, if the position and shooting angle of the first image acquisition device are adjusted, the calibration parameters of the first image acquisition device can be recorded after the adjustment is completed. These calibration parameters may include intrinsic parameters (such as focal length), extrinsic parameters (such as position and shooting angle), and distortion parameters of the first image acquisition device.

[0084] It is understandable that each time the host computer acquires the first image acquired by the first image acquisition device, it can also acquire the calibration parameters of the first image acquisition device when acquiring that first image.

[0085] S602. The first images are processed by a preset surround view algorithm to obtain a first surround view image.

[0086] Among them, the surround view algorithm is an image processing technology that can fuse images captured by multiple cameras into a panoramic image.

[0087] Specifically, based on the surround view algorithm, the host computer can stitch and fuse the first images of the model car from different perspectives captured by multiple first image acquisition devices to generate a surround view image of the model car.

[0088] In the above embodiments, by acquiring first images taken by multiple first image acquisition devices at different positions and angles, the host computer can use a surround-view algorithm to generate surround-view images of the model car, which can then be used for testing the AVM system. This method eliminates the need for a real car; by using a model car instead, AVM testing costs can be reduced, and the difficulty of acquiring images at different positions and angles can be lowered, thus improving image acquisition efficiency.

[0089] In one embodiment, processing multiple first images using a preset surround-view algorithm to obtain a first surround-view image includes:

[0090] Obtain the calibration parameters of the first image acquisition device when acquiring the first image; process the first image according to the calibration parameters and the preset perspective transformation matrix, and stitch the processed images of multiple first images into a first panoramic image.

[0091] The calibration parameters may include intrinsic parameters (such as focal length), extrinsic parameters (such as position and shooting angle), and distortion parameters of the first image acquisition device. The perspective transformation matrix is ​​used to convert the first image from the shooting perspective of the first image acquisition device into a top-down view.

[0092] Specifically, feature points can be selected in the first image, and a first coordinate system can be established based on these feature points. Then, the mapping relationship between the first coordinate system and a preset reference coordinate system can be determined according to calibration parameters. Based on this mapping relationship, the coordinates of the pixels in the first image in the reference coordinate system can be obtained, thus converting the first image into a top-down view image. By stitching together the converted images from each first image, a first panoramic top-down view image can be obtained. The reference coordinate system can be a coordinate system established based on the top-down view.

[0093] In one embodiment, a marker is set around the model car, and acquiring first images of the model car in a preset direction captured by multiple first image acquisition devices includes: acquiring a first image including the marker captured by the first image acquisition device. The vehicle image processing method further includes:

[0094] Obtain a top-view image containing the calibration object; optimize the target parameters in the surround view algorithm based on the positional differences of the calibration object in the first surround view image and the top-view image.

[0095] The target parameters include the calibration parameters of the first image acquisition device and / or the parameters of the perspective transformation matrix.

[0096] In some possible implementations, the calibration object can be a calibration plate set around the model car. The calibration plate can have multiple corner points.

[0097] Specifically, a top-down image containing the calibration objects around the model vehicle can be acquired using a second image acquisition device. The top-down image is used as ground truth data, and corner detection algorithms (such as Harris corner detection or sub-pixel corner detection) can be used to identify the corners on the calibration plate in the top-down image and the first surround view image respectively. The differences of the same corners in the top-down image and the first surround view image are compared to improve the target parameters related to the surround view algorithm based on the differences, so that the first surround view image generated by the host computer is closer to the ground truth data.

[0098] The second image acquisition device may be the second visual sensor 7 in the above embodiments.

[0099] For example, the method of acquiring top-down images using a second image acquisition device can be referred to Figure 5 The second image acquisition device can be set above the model car, so that the shooting range of the second image acquisition device covers the model car and the surrounding landmarks.

[0100] In some possible implementations, the calibration object has multiple calibration points. Based on the positional differences of the calibration object in the first panoramic image and the top-view image, the target parameters in the panoramic algorithm are optimized, including:

[0101] Determine the positional offset of the same calibration point on the calibration object in the first panoramic image and the top-view image; adjust the target parameters to reduce the positional offset.

[0102] Specifically, the position of the same calibration point on the calibration object in the first panoramic image and its position in the panoramic image can be transformed to the same coordinate system, and the offset between the two positions can be calculated based on the coordinate system.

[0103] After obtaining the offset, a loss function can be established based on the position offset and the target parameter. The target parameter corresponding to the position offset being less than the preset standard can be obtained by iteratively applying the gradient descent algorithm.

[0104] For example, the independent variable of the loss function can be the target parameter, and the function value of the loss function can be the position offset of the same calibration point. The target parameter can be continuously adjusted to reduce the position offset until the position offset is less than a preset threshold.

[0105] In other possible examples, parameters such as the sum, average, and standard deviation of the positional offsets of multiple corner points can also be used as the optimization objective of the loss function. The optimization objective can be made to meet the preset standard by adjusting the objective parameters.

[0106] Figure 7 This is a schematic diagram of the vehicle image processing device provided in this application. Figure 7As shown, the vehicle image processing device 700 provided in this embodiment may include:

[0107] The acquisition module 701 is used to acquire first images of the model vehicle in a preset direction, which are acquired by multiple first image acquisition devices.

[0108] The image processing module 702 is used to process multiple first images using a preset surround view algorithm to obtain a first surround view image.

[0109] In one embodiment, the image processing module 702 may include:

[0110] The calibration acquisition module can be used to acquire calibration parameters when the first image acquisition device acquires the first image.

[0111] The image stitching module can be used to process the first image according to the calibration parameters and the preset perspective transformation matrix, and stitch the processed images of multiple first images into a first panoramic image. The perspective transformation matrix is ​​used to convert the first image from the shooting perspective of the first image acquisition device to a top-down perspective.

[0112] In one embodiment, a marker is provided around the model vehicle, and the acquisition module 701 is further configured to acquire a first image including the marker captured by the first image acquisition device. The vehicle image processing device 700 further includes:

[0113] The top-view acquisition module can be used to acquire top-view images containing the calibration object.

[0114] The parameter optimization module can be used to optimize the target parameters in the surround view algorithm based on the positional differences of the calibration object in the first surround view image and the top view image. The target parameters include the calibration parameters of the first image acquisition device and / or the parameters of the perspective transformation matrix.

[0115] In one embodiment, the calibration object has multiple calibration points, and the parameter optimization module includes:

[0116] The offset determination unit can be used to determine the positional offset of the same calibration point on the calibration object in the first panoramic image and the top-view image.

[0117] The parameter adjustment unit can be used to adjust the target parameters to reduce the position offset.

[0118] The vehicle image processing device provided in this embodiment is used to execute the technical solutions in any of the aforementioned method embodiments. Its implementation principle and technical effects are similar, and will not be described again here.

[0119] It should be understood that the above-described device embodiments are merely illustrative, and the device of the present invention can also be implemented in other ways. For example, the division of units / modules in the above embodiments is only a logical functional division, and there may be other division methods in actual implementation. For example, multiple units, modules, or components may be combined, or integrated into another system, or some features may be ignored or not executed.

[0120] Furthermore, unless otherwise specified, the functional units / modules in the various embodiments of the present invention can be integrated into one unit / module, or each unit / module can exist physically separately, or two or more units / modules can be integrated together. The integrated units / modules described above can be implemented in hardware or as software program modules.

[0121] Figure 8 This is a schematic diagram of the structure of an electronic processing device provided in this application. Figure 8 As shown, the electronic processing device provided in this embodiment is an electronic device 80, which includes:

[0122] Processor 81, memory 82, and communication interface 83;

[0123] The memory 82 is used to store the executable instructions of the processor 81; the executable instructions may be computer-executable instructions.

[0124] The processor 81 is configured to execute the technical solutions in any of the foregoing method embodiments by executing the executable instructions.

[0125] Optionally, the memory 82 can be either standalone or integrated with the processor 81.

[0126] Optionally, when the memory 82 is a device independent of the processor 81, the electronic device 80 may further include:

[0127] Bus 84, memory 82 and communication interface 83 are connected to processor 81 through bus 84 and complete communication with each other. Communication interface 83 is used to communicate with other devices.

[0128] Optionally, the communication interface 83 can be implemented using a transceiver. The communication interface is used to enable communication between the database access device and other devices (e.g., clients, read-write databases, and read-only databases). The memory may include random access memory (RAM) and may also include non-volatile memory, such as at least one disk storage device.

[0129] Bus 84 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of representation, only one line is used in the diagram, but this does not imply that there is only one bus or one type of bus.

[0130] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0131] The electronic device is used to execute the technical solutions in any of the foregoing method embodiments. Its implementation principle and technical effect are similar, and will not be described again here.

[0132] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0133] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the above-described method.

[0134] The aforementioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0135] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in the device.

[0136] The division of units is merely a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0137] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0138] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0139] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0140] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0141] Finally, it should be noted that other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This invention is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein, and is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.

Claims

1. A vehicle image processing system, characterized in that, Including the host computer and model car; The model vehicle is equipped with a controller and a fixed structure for deploying multiple first vision sensors, the fixed structure allowing the first vision sensors to adjust their position and shooting angle; The host computer is connected to the controller and is used to send a first control command to the controller; The host computer is also used to acquire the first image collected by the first vision sensor and generate a surround view image corresponding to the model vehicle based on the first image. The controller is used to drive the model car to move under the action of the first control command.

2. The system according to claim 1, characterized in that, The fixing structure includes multiple bases disposed on the side of the model vehicle, each base being used to place at least one of the first vision sensors.

3. The system according to claim 1, characterized in that, The fixing structure includes a clamp and a support tube or guide rail disposed on the side of the model vehicle, and the first vision sensor is fixed to the support tube or the guide rail by the clamp.

4. The system according to any one of claims 1 to 3, characterized in that, The host computer is also used to send a second control command to the first vision sensor, the second control command being used to control the first vision sensor to adjust its position and / or shooting angle on the model vehicle.

5. The system according to any one of claims 1 to 3, characterized in that, The vehicle monitoring image processing system also includes a display installed on the model vehicle. The display is connected to the host computer and is used to receive and display images transmitted by the host computer.

6. The system according to any one of claims 1 to 3, characterized in that, The vehicle image processing system also includes a second vision sensor connected to the host computer, which is used to acquire top-down images of the model vehicle.

7. A vehicle image processing method, characterized in that, Applied to a host computer as described in any one of claims 1 to 6, the method comprises: Acquire first images of the model vehicle in a preset direction, captured by multiple first image acquisition devices; The first panoramic image is obtained by processing multiple first images using a preset panoramic algorithm.

8. The method according to claim 7, characterized in that, The step of processing multiple first images using a preset surround-view algorithm to obtain a first surround-view image includes: Obtain the calibration parameters of the first image acquisition device when acquiring the first image; The first image is processed according to the calibration parameters and the preset perspective transformation matrix, and the processed images of multiple first images are stitched together to form a first panoramic image. The perspective transformation matrix is ​​used to convert the first image from the shooting angle of the first image acquisition device to a top-down view.

9. The method according to claim 8, characterized in that, The model vehicle is surrounded by markers, and the acquisition of first images of the model vehicle in a preset direction, captured by multiple first image acquisition devices, includes: Acquire a first image containing the calibration object captured by the first image acquisition device; The method further includes: Acquire a top-view image including the calibration object; Based on the positional differences of the calibration object in the first panoramic image and the top-view image, the target parameters in the panoramic algorithm are optimized. The target parameters include the calibration parameters of the first image acquisition device and / or the parameters of the perspective transformation matrix.

10. The method according to claim 9, characterized in that, The calibration object has multiple calibration points. Optimizing the target parameters in the surround view algorithm based on the positional differences of the calibration object in the first surround view image and the top view image includes: Determine the positional offset of the same calibration point on the calibration object in the first panoramic image and the top-view image; Adjust the target parameters to reduce the position offset.

11. A vehicle image processing device, characterized in that, include: The acquisition module is used to acquire first images of the model vehicle in a preset direction, acquired by multiple first image acquisition devices; The image processing module is used to process multiple first images using a preset surround view algorithm to obtain a first surround view image.

12. An electronic processing device, characterized in that, include: Memory, processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory, causing the processor to perform the method as described in any one of claims 7 to 10.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 7 to 10.

14. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method of any one of claims 7 to 10.

Images