Image acquisition method, device and system

By adjusting the angle and exposure time of the camera module, combined with optical and electronic anti-shake technology, the problem of image jitter outside the cabin during and after driving is solved, the image acquisition effect and image quality are improved, and the user's needs for high-quality images are met.

CN119031246APending Publication Date: 2024-11-26BEIJING CO WHEELS TECH CO LTD
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202310611456.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-05-26
Publication Date
2024-11-26

AI Technical Summary

Technical Problem

During the car driving, the quality of the out-of-cabin images is not high due to factors such as acceleration, deceleration and vibration, which cannot meet the actual needs of users.

Method used

By receiving the vehicle's motion sensor data and controller data, adjusting the angle and exposure time of the camera module, combining optical anti-shake and electronic anti-shake technology, stable and high-quality off-cabin images are collected.

Benefits of technology

It effectively solves the problem of image jitter outside the cabin when driving a car, improves the image acquisition effect and image quality, and enhances the image experience of the car's smart cockpit.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119031246A_ABST
    Figure CN119031246A_ABST
Patent Text Reader

Abstract

The invention provides an image acquisition method, device and system, and relates to the technical field of image acquisition. The method comprises the steps that first data and second data are received, the first data comprise data collected by a motion sensor of a vehicle, and the second data comprise data obtained by a controller of the vehicle; according to the first data and / or the second data, determining an adjustment angle of a camera module of the vehicle so as to adjust the angle of the camera module of the vehicle; adjusting the exposure time of a camera module of the vehicle to the first exposure time corresponding to the second data; the first image is collected through the camera module, the first data and the second data are combined, the problem that images collected outside a cabin shake in the moving state of a running automobile is solved, the image collection effect of the automobile is improved, and the image quality of the collected images is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to the field of image acquisition technology, and in particular to an image acquisition method, device and system. Background Art

[0002] With the popularization of vehicles and the upgrade of hardware, the current demands of car users for vehicles have also changed from basic driving to the pursuit of high-quality driving experience. Simple in-car entertainment can no longer meet user needs, and sharing of life content around cars has become increasingly important. In order to bring users the ultimate audio-visual experience in the cockpit, sometimes the information inside and outside the car cockpit is multimodally fused, which requires projecting the image outside the car cabin into the car cockpit. However, due to acceleration, deceleration, vibration and other reasons during the driving of the car, the quality of the obtained image outside the cabin is not high and cannot meet the actual needs; therefore, how to obtain the image outside the cabin that meets the actual needs during the driving of the vehicle has become an urgent problem to be solved. Summary of the invention

[0003] The present disclosure provides an image acquisition method, device, system, electronic device and storage medium.

[0004] According to a first aspect of the present disclosure, there is provided an image acquisition method, which is applied to a vehicle, and the method comprises: receiving first data and second data, the first data comprising data acquired by a motion sensor of the vehicle, and the second data comprising data acquired by a controller of the vehicle; determining an adjustment angle of a camera module of the vehicle through the first data and / or the second data to adjust the angle of the camera module of the vehicle; adjusting the exposure time of the camera module of the vehicle to a first exposure time corresponding to the second data; and acquiring a first image through the camera module.

[0005] In some embodiments, the adjustment angle of the camera module of the vehicle is determined through the first data and / or the second data, and adjusting the angle of the camera module of the vehicle includes: determining the movement amplitude and movement direction of the vehicle through the first data and / or the second data; determining the adjustment angle of the camera module of the vehicle based on the movement amplitude and movement direction; controlling the vehicle's driver to send an adjustment instruction to the electric motor of the camera module, the adjustment instruction including the adjustment angle; determining the moving direction corresponding to the adjustment instruction; and adjusting the camera module to the moving direction.

[0006] In some embodiments, adjusting the exposure time of the vehicle's camera module to a first exposure time corresponding to second data includes: determining a first corresponding relationship based on a linear relationship between multiple second data and multiple first exposure times; and determining the first exposure time corresponding to the second data based on the first corresponding relationship and the second data.

[0007] In some embodiments, the method also includes: preprocessing the first image to obtain a second image, wherein the preprocessing includes at least one of format conversion and image quality optimization; processing the first data and the second data through a preset algorithm to obtain the motion state of the vehicle; and determining the field of view of the camera module according to the motion state to adjust the display range of the second image.

[0008] In some embodiments, there are one or more camera modules, and the method further includes: when there are multiple camera modules, stitching the output images of each camera module to obtain a stitched image.

[0009] According to an embodiment of the present disclosure, by receiving first data and second data, the first data includes data collected by a motion sensor of the vehicle, and the second data includes data acquired by a controller of the vehicle; the adjustment angle of the camera module of the vehicle is determined by the first data and / or the second data to adjust the angle of the camera module of the vehicle; the exposure time of the camera module of the vehicle is adjusted to the first exposure time corresponding to the second data; the first image is collected by the camera module to combine the first data with the second data, thereby solving the problem of jitter of images collected outside the cabin when the moving car is in motion, improving the image collection effect of the car, and improving the image quality of the collected images.

[0010] According to a second aspect of the present disclosure, there is provided an image acquisition device, which is configured in a vehicle, and includes: a receiving module, used to receive first data and second data, the first data including data acquired by a motion sensor of the vehicle, and the second data including data acquired by a controller of the vehicle; a first adjustment module, used to determine an adjustment angle of a camera module of the vehicle through the first data and / or the second data, so as to adjust the angle of the camera module of the vehicle; a second adjustment module, used to adjust the exposure time of the camera module of the vehicle to a first exposure time corresponding to the second data; and an acquisition module, used to acquire a first image through the camera module.

[0011] According to an embodiment of the present disclosure, by receiving first data and second data, the first data includes data collected by a motion sensor of the vehicle, and the second data includes data acquired by a controller of the vehicle; the adjustment angle of the camera module of the vehicle is determined by the first data and / or the second data to adjust the angle of the camera module of the vehicle; the exposure time of the camera module of the vehicle is adjusted to the first exposure time corresponding to the second data; the first image is collected by the camera module to combine the first data with the second data, thereby solving the problem of jitter of images collected outside the cabin when the moving car is in motion, improving the image collection effect of the car, and improving the image quality of the collected images.

[0012] According to a third aspect of the present disclosure, an image acquisition system is provided, which is applied to a vehicle, and the system includes: a camera module; the camera module includes one or more cameras and an image acquisition device, the image acquisition device is used to adjust the angle of one or more cameras based on first data collected by a motion sensor of the vehicle and second data sent by a controller of the vehicle, and adjust the first exposure time of the camera module of the vehicle to a second exposure time corresponding to the second data.

[0013] In some embodiments, the image acquisition device includes an electric motor and a driver, the driver is connected to the electric motor and the motion sensor respectively, and the driver is used to control the movement of the electric motor to adjust the angle of the camera module.

[0014] In some embodiments, the image acquisition device is wirelessly connected to a controller of the vehicle for receiving the second data.

[0015] According to the embodiments of the present disclosure, through the image acquisition system, the camera module can be adjusted to combine the first data with the second data to solve the problem of jitter of the image captured outside the cabin when the car is in motion, thereby improving the image acquisition effect of the car and the quality of the captured image.

[0016] According to a fourth aspect of the present disclosure, there is provided an electronic device, including:

[0017] at least one processor; and

[0018] a memory communicatively connected to at least one processor; wherein,

[0019] The memory stores instructions that can be executed by at least one processor, and the instructions are executed by at least one processor to enable the at least one processor to execute the method of the first aspect described above.

[0020] According to a fifth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable a computer to execute the method of the aforementioned first aspect.

[0021] According to a sixth aspect of the present disclosure, a vehicle is provided, comprising the image acquisition system as described in the third aspect or the electronic device as described in the fourth aspect.

[0022] In some embodiments, the vehicle further includes: a motion sensor, wherein the motion sensor includes a gyroscope and an acceleration sensor, and the motion sensor is used to collect the first data.

[0023] According to an embodiment of the present disclosure, by receiving first data and second data, the first data includes data collected by a motion sensor of the vehicle, and the second data includes data acquired by a controller of the vehicle; the adjustment angle of the camera module of the vehicle is determined by the first data and / or the second data to adjust the angle of the camera module of the vehicle; the exposure time of the camera module of the vehicle is adjusted to the first exposure time corresponding to the second data; the first image is collected by the camera module to combine the first data with the second data, thereby solving the problem of jitter of images collected outside the cabin when the moving car is in motion, improving the image collection effect of the car, and improving the image quality of the collected images.

[0024] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0026] Figure 1 A flowchart of an image acquisition method provided by an embodiment of the present disclosure;

[0027] Figure 2 A flowchart of an image acquisition method provided by an embodiment of the present disclosure;

[0028] Figure 3 A schematic diagram of angle compensation for vehicle OIS anti-shake provided in an embodiment of the present disclosure;

[0029] Figure 4 A schematic diagram of an OIS control algorithm provided in an embodiment of the present disclosure;

[0030] Figure 5 A schematic diagram of adjusting exposure time to reduce motion blur provided by an embodiment of the present disclosure;

[0031] Figure 6 A schematic diagram of an EIS anti-shake algorithm provided in an embodiment of the present disclosure;

[0032] Figure 7 A flowchart of a specific image acquisition method provided by an embodiment of the present disclosure;

[0033] Figure 8 A schematic diagram of an image acquisition system provided by an embodiment of the present disclosure;

[0034] Fig. 9 A schematic diagram of the structure of an image acquisition device provided in an embodiment of the present disclosure;

[0035] Fig.10A schematic block diagram of an exemplary electronic device 1000 provided for an embodiment of the present disclosure. DETAILED DESCRIPTION

[0036] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0037] An image acquisition method, device, system, electronic device, and storage medium according to an embodiment of the present disclosure are described below with reference to the accompanying drawings.

[0038] In response to the problems of the above-mentioned existing technologies, the image acquisition method can be used to better obtain the vehicle's motion state, improve the fusion of multi-sensory signals, solve the problem of jitter of images collected outside the cabin when the car is in motion through optical image stabilization, improve image quality, enhance the imaging experience of the car's smart cockpit and the multimodal fusion of internal and external cockpit images.

[0039] The image acquisition method proposed in the present disclosure can be applied to a device with an image acquisition function. The application scenario is not limited in the embodiments of the present disclosure. The present disclosure mainly takes the vehicle-mounted image acquisition scenario as an example.

[0040] The following describes in detail an image acquisition method, device, electronic device, storage medium, and program product proposed in the present disclosure in conjunction with the accompanying drawings.

[0041] Figure 1 The following is a flow chart of an image acquisition method provided by an embodiment of the present disclosure. Figure 1 As shown, the method is applied to a vehicle, and the method comprises the following steps:

[0042] Step 101 , receiving first data and second data, wherein the first data includes data collected by a motion sensor of a vehicle, and the second data includes data acquired by a controller of the vehicle.

[0043] In one implementation of the present disclosure, the motion sensor may include a vehicle-mounted gyroscope (gyro) and an accelerometer (acc), wherein the vehicle-mounted gyroscope is used to measure the rotation angle and movement speed of the vehicle, and the accelerometer is used to measure the acceleration and speed of the vehicle. The vehicle controller may be the vehicle's ECU (electronic controller unit of the vehicle), which is used to obtain the vehicle's motion information during driving, and the motion information may be the vehicle's speed, acceleration, rotation information, pedal information, etc. For example, when the vehicle turns the steering wheel, the vehicle's ECU detects the direction of rotation of the steering wheel and sends the rotation method to the image acquisition device.

[0044] The present disclosure receives the first data and the second data through a camera module of the vehicle. The camera module can be located outside the vehicle or inside the vehicle, which is not limited in the embodiments of the present disclosure.

[0045] Step 102: determining an adjustment angle of the camera module of the vehicle by using the first data and / or the second data to adjust the angle of the camera module of the vehicle.

[0046] In one embodiment of the present disclosure, after receiving the first data and / or the second data, the camera module can obtain the adjustment angle of the camera module according to the first data and / or the second data, and then adjust the camera module in the image acquisition device, that is, the angle of the camera module can be adjusted to compensate for the offset angle caused by the shaking of the vehicle, so as to achieve hardware anti-shake. The adjustment of the angle of the camera module is mainly achieved by the driver and the electric motor in the camera module.

[0047] Step 103, adjusting the exposure time of the camera module of the vehicle to the first exposure time corresponding to the second data.

[0048] In one embodiment of the present disclosure, the camera module adjusts the exposure time of the camera module to the first exposure time according to the second data. Since the camera module can sample a high frame rate wide-angle camera sensor component, the exposure time can be accurately adjusted to the first exposure time according to the second data during the sensor sensing period to reduce motion blur generated during the vehicle driving process. The first exposure time refers to the exposure time corresponding to the current second data.

[0049] It should be noted that there can be multiple or one camera module, and the number of camera modules and the installation position can be set according to actual needs and are not limited in the embodiments of the present disclosure. In the present disclosure, the number of camera modules can be set to 4, that is, one camera module is installed in each of the front, rear, left and right directions of the vehicle.

[0050] Step 104: Capture a first image through a camera module.

[0051] In one embodiment of the present disclosure, the image outside the vehicle cabin is captured by the adjusted camera module, and the image outside the vehicle cabin is the first image.

[0052] In addition, in the present disclosure, it is necessary to perform image processing on the first image, wherein the image processing includes electronic image stabilization on the first image, so that the output image can meet better quality requirements.

[0053] Therefore, according to an embodiment of the present disclosure, by receiving first data and second data, the first data includes data collected by the vehicle's motion sensor, and the second data includes data acquired by the vehicle's controller; the adjustment angle of the vehicle's camera module is determined by the first data and / or the second data to adjust the angle of the vehicle's camera module; the exposure time of the vehicle's camera module is adjusted to the first exposure time corresponding to the second data; the first image is collected by the camera module to combine the first data with the second data, thereby solving the problem of jitter of images collected outside the cabin when the moving car is in motion, improving the image collection effect of the car, and improving the image quality of the collected images.

[0054] Figure 2 A flowchart of an image acquisition method provided in an embodiment of the present disclosure. Figure 2 based on Figure 1 In the embodiment shown, step 104 is further defined. Figure 2 In the embodiment shown, step 104 is followed by step 204. Figure 2 As shown, the method includes the following steps.

[0055] Step 201: Receive first data and second data.

[0056] In some implementations of the present disclosure, step 201 has the same or similar functions as the above step 101, and its implementation can be performed with reference to the above embodiment, which will not be described in detail here.

[0057] Step 202: Determine the adjustment angle of the camera module of the vehicle through the first data and / or the second data to adjust the angle of the camera module of the vehicle.

[0058] In some embodiments of the present disclosure, the movement amplitude and movement direction of the vehicle can be determined through the first data and / or the second data; based on the movement amplitude and the movement direction, the adjustment angle of the camera module of the vehicle can be determined; the driver of the vehicle is controlled to send an adjustment instruction to the electric motor of the camera module, the adjustment instruction including the adjustment angle; the moving direction corresponding to the adjustment instruction is determined; and the camera module is adjusted to the moving direction.

[0059] Adjusting the angle of the vehicle's camera module is the process of hardware image stabilization, i.e., OIS optical image stabilization, which is to install a micro electric motor inside the lens that can move in multiple directions. The image acquisition device can convert the first data monitored by the gyroscope and the accelerometer and the second data acquired by the vehicle controller into electrical signals. The driver in the camera module will determine the current tilt angle based on these data, predict the image offset caused by the tilt based on the angle, determine the adjustment angle based on the image offset, and then feed the result back to the electric motor, send an adjustment instruction, and control the electric motor to push the camera module to move with a displacement of the same size but opposite direction as the predicted image offset, thereby offsetting the image offset caused by some shaking.

[0060] like Figure 3 , Figure 4 The figure shows the angle compensation diagram of the vehicle OIS anti-shake and the OIS control algorithm diagram. The OIS optical anti-shake solves the problem of large-angle image quality jitter during vehicle driving, and combines the first data with the second data to achieve precise drive control of the algorithm. For example, the speed of the moving vehicle, as well as vehicle braking or starting signals, can be obtained as algorithm parameter inputs to better improve the OIS-assisted image stabilization effect.

[0061] It is understandable that by adjusting the angle of the camera module, part of the angle deviation caused by the movement of the vehicle can be compensated, and the remaining angle compensation is achieved through electronic image stabilization, that is, the method in step 204 below.

[0062] Step 203, adjusting the exposure time of the camera module of the vehicle to the first exposure time corresponding to the second data.

[0063] In some embodiments of the present disclosure, exposure time refers to the time a pixel is exposed to light, which affects the number of electrons in the potential well and thus affects the brightness of the pixel, and is usually measured in ms.

[0064] like Figure 5 As shown, when the vehicle is moving, the exposure time must be fast enough to make the vehicle relatively stationary relative to the camera module. Otherwise, if the exposure time is too slow, the vehicle is relatively moving relative to the camera module, and the captured image is prone to blurring and ghosting.

[0065] In the present disclosure, adjusting the exposure time includes: determining a first corresponding relationship based on a linear relationship between a plurality of second data and a plurality of first exposure times; and determining the first exposure time corresponding to the second data according to the first corresponding relationship and the second data.

[0066] The first corresponding relationship may be a linear relationship (linear function) between the first exposure time and the speed v established according to the speed or acceleration of the vehicle. For example, the linear relationship between the first exposure time and the speed v may be: Wherein, Time represents the first exposure time, v represents the speed, and k is a constant. The camera module can obtain the first exposure time corresponding to the current second data according to the linear relationship and the second data currently acquired by the vehicle controller to collect the first image.

[0067] It should be noted that the order of adjusting the exposure time of the camera module and the angle of the camera module is not limited in the embodiments of the present disclosure. The order in the present disclosure is only used as an example. It is also possible to adjust the angle of the camera module first and then adjust the exposure time of the camera module.

[0068] Step 204, based on the motion state of the vehicle, adjust the size of the first image captured by the camera module to obtain an output image.

[0069] In some embodiments of the present disclosure, the first image is preprocessed to obtain the second image, wherein the preprocessing includes at least one of format conversion and image quality optimization; the first data and the second data are processed by a preset algorithm to obtain the motion state of the vehicle; based on the motion state, the field of view of the camera module is determined, and the field of view is used to adjust the display range of the second image.

[0070] Specifically, the preprocessing of the first image includes performing distortion correction on the first image. The lens may introduce distortion due to deviations in manufacturing precision and assembly process, which may easily lead to distortion of the original image. At this time, the LDC (Lens Distortion Correction) distortion algorithm may be used to correct the image distortion. The distortion of the lens is divided into two categories: radial distortion and tangential distortion.

[0071] At the same time, the first image preprocessing also includes color correction, downsampling, de-mosaicing and statistical 3A data of the first image through IFE (Image Front End), and hardware noise reduction (MFNR, MFSR), image cropping, noise reduction, color processing, detail enhancement and other image processing through IPE (Image processing engine).

[0072] It can be understood that the second image is the image of the first image after the image quality is enhanced. The preset algorithm is an algorithm for image stabilization, such as the EIS electronic image stabilization algorithm. The EIS electronic image stabilization algorithm is a software compensation algorithm. The principle is like "post-processing" of imaging. It analyzes and collects the image on the sensor through an algorithm, and dynamically adjusts the ISO, shutter, etc. according to the degree of body shaking to make blur correction.

[0073] like Figure 6 As shown, it is a schematic diagram of the eis anti-shake algorithm. The eis video image anti-shake algorithm can compensate for the remaining image offset caused by the movement of the vehicle, and is used to perform anti-shake algorithm processing on each frame of RAW data in the image collected outside the cabin. The first data and the second data can be combined to predict the vehicle's motion state at the next moment. Specifically, the data collected by the on-board gyroscope and accelerometer and the vehicle's instantaneous speed, acceleration, vehicle start or braking signals, etc. obtained by the vehicle's controller can be input into the image anti-shake algorithm to predict the vehicle's motion state at the next moment, so that the video image anti-shake effect is smoother. For example, through the steering data obtained by the vehicle controller, the anti-shake algorithm predicts the direction of movement of the vehicle at the next moment.

[0074] In an optional embodiment of the present disclosure, there may be one or more camera modules. When there are multiple camera modules, the output images of the various camera modules are spliced ​​to obtain a spliced ​​image. The spliced ​​image can project the image around the moving vehicle 360 ​​degrees to enhance the user's cockpit stereoscopic visual experience.

[0075] In addition, the present disclosure can also increase the in-vehicle intelligent cockpit image entertainment function to enhance the user's image entertainment experience of the space outside the cabin of the moving car.

[0076] In summary, according to the embodiments of the present disclosure, first data and second data are received, and the adjustment angle of the camera module of the vehicle is determined through the first data and / or the second data to adjust the angle of the camera module of the vehicle; the exposure time of the camera module of the vehicle is adjusted to the first exposure time corresponding to the second data, and based on the motion state of the vehicle, the size of the first image captured by the camera module is adjusted to obtain an output image. By inputting the vehicle speed / acceleration and vehicle start / braking signal data into the algorithm, the algorithm's prediction accuracy of the vehicle's motion state can be improved, and the anti-shake effect of the entire algorithm is improved. Combined with video electronic anti-shake and optical anti-shake technology, the problem of shaking when capturing images outside the cabin of a moving car is solved, the image capture effect of the car is improved, and the image quality of the captured image is improved. At the same time, the surround image technology improves the visual experience of projecting the image outside the cabin of the moving car to the image space inside the cabin.

[0077] based on Figure 1 and Figure 2The proposed image acquisition method, the present disclosure provides a specific flow chart of the image acquisition method, such as Figure 7 shown.

[0078] In some embodiments of the present disclosure, the first data and the second data are received by a motion sensor of the vehicle and a controller of the vehicle, the angle and the exposure time of the camera module are adjusted by the OIS optical image stabilization on the gimbal end, and the first image is captured by the adjusted camera module.

[0079] In some embodiments of the present disclosure, the first image is preprocessed by digital ISP image front-end processing IFE and digital ISP image back-end processing IPE and LDC distortion correction to obtain a second image. The second image is electronically image-stabilized by the EIS anti-shake algorithm, that is, the field of view of the second image is adjusted to obtain an output image.

[0080] In some embodiments of the present disclosure, when there are multiple camera modules, the output images acquired by the multiple camera modules may be stitched to effect surround view.

[0081] In summary, projecting 360-degree images outside the cabin into the cabin enhances the technological sense of the smart cockpit image space. At the same time, the improvement and use of electronic image stabilization and optical image stabilization in the smart cockpit greatly improves the image quality of the images collected during vehicle driving, and provides an image data foundation for the visual fusion inside and outside the smart cockpit. Among them, the improvement and application of the anti-shake algorithm greatly improves the image quality during vehicle driving, and provides an image quality foundation for the multi-modal visual fusion inside and outside the smart cockpit.

[0082] based on Figure 1 and Figure 2 The proposed image acquisition method, the present disclosure provides an image acquisition system 800, such as Figure 8 As shown, the system is applied to a vehicle, and the system includes:

[0083] Camera module 810;

[0084] The camera module 810 includes one or more cameras 811 and an image acquisition device 812. The image acquisition device is used to adjust the angle of one or more cameras based on the first data collected by the vehicle's motion sensor and the second data sent by the vehicle's controller, and adjust the first exposure time of the vehicle's camera module to the second exposure time corresponding to the second data.

[0085] In some embodiments of the present disclosure, the image acquisition device 812 includes an electric motor and a driver, the driver is connected to the electric motor and the motion sensor respectively, and the driver is used to control the movement of the electric motor to adjust the angle of the camera module.

[0086] The camera is used to collect the impact outside the vehicle cabin, that is, the first image. The image acquisition device is used for hardware anti-shake, that is, OIS optical anti-shake gimbal. Multiple cameras can share one image acquisition device.

[0087] In some embodiments of the present disclosure, the image acquisition device 812 is wirelessly connected to a controller of the vehicle for receiving the second data.

[0088] In summary, through the image acquisition system, adjusting the camera module can solve the problem of jitter of the image captured outside the cabin when the car is in motion by combining the first data with the second data, thereby improving the image acquisition effect of the car and improving the image quality of the captured image.

[0089] Corresponding to the above-mentioned image acquisition method, the present disclosure also proposes an image acquisition device. Fig. 9 FIG. 9 is a schematic diagram of the structure of an image acquisition device 900 provided in an embodiment of the present disclosure. Figure 4 As shown, the device is configured in a vehicle, and the device includes:

[0090] A receiving module 910, configured to receive first data and second data, wherein the first data includes data collected by a motion sensor of the vehicle, and the second data includes data acquired by a controller of the vehicle;

[0091] A first adjustment module 920, configured to determine an adjustment angle of a camera module of the vehicle through the first data and / or the second data, so as to adjust an angle of the camera module of the vehicle;

[0092] A second adjustment module 930, configured to adjust the exposure time of the camera module of the vehicle to a first exposure time corresponding to the second data;

[0093] The acquisition module 940 is used to acquire a first image through a camera module.

[0094] In some embodiments, the first adjustment module 920 is used to: determine the movement amplitude and movement direction of the vehicle through the first data and / or the second data; determine the adjustment angle of the vehicle's camera module based on the movement amplitude and movement direction; control the vehicle's driver to send adjustment instructions to the electric motor of the camera module, the adjustment instructions including adjusting the angle; determine the moving direction corresponding to the adjustment instruction; and adjust the camera module to the moving direction.

[0095] In some embodiments, the second adjustment module 930 is used to: determine a first corresponding relationship based on a linear relationship between a plurality of second data and a plurality of first exposure times; and determine a first exposure time corresponding to the second data according to the first corresponding relationship and the second data.

[0096] In some embodiments, the device 900 also includes: a processing module, used to pre-process the first image to obtain a second image, wherein the pre-processing includes at least one of format conversion and image quality optimization; processing the first data and the second data through a preset algorithm to obtain the motion state of the vehicle; determining the field of view of the camera module according to the motion state, and the field of view is used to adjust the display range of the second image.

[0097] In some embodiments, the device 900 further includes: a stitching module, which is used for stitching output images of each camera module to obtain a stitched image when there are one or more camera modules.

[0098] According to an embodiment of the present disclosure, by receiving first data and second data, the first data includes data collected by a motion sensor of the vehicle, and the second data includes data acquired by a controller of the vehicle; the adjustment angle of the camera module of the vehicle is determined by the first data and / or the second data to adjust the angle of the camera module of the vehicle; the exposure time of the camera module of the vehicle is adjusted to a first exposure time corresponding to the second data; the first image is collected by the camera module to combine the first data with the second data, thereby solving the problem of jitter of images collected outside the cabin when the moving car is in motion, improving the image collection effect of the car, and improving the image quality of the collected images.

[0099] It should be noted that since the device embodiment of the present disclosure corresponds to the above-mentioned method embodiment, the above-mentioned explanation of the method embodiment is also applicable to the device of the present embodiment, and the principle is the same. For details not disclosed in the device embodiment, reference can be made to the above-mentioned method embodiment, and they will not be repeated in this disclosure.

[0100] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0101] Fig.10 A schematic block diagram of an example electronic device 1000 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0102] like Fig.10As shown, the device 1000 includes a computing unit 1001, which can perform various appropriate actions and processes according to a computer program stored in a ROM (Read-Only Memory) 1002 or a computer program loaded from a storage unit 1008 to a RAM (Random Access Memory) 1003. In the RAM 1003, various programs and data required for the operation of the device 1000 can also be stored. The computing unit 1001, the ROM 1002, and the RAM 1003 are connected to each other via a bus 1004. An I / O (Input / Output) interface 1005 is also connected to the bus 1004.

[0103] A number of components in the device 1000 are connected to the I / O interface 1005, including: an input unit 1006, such as a keyboard, a mouse, etc.; an output unit 1007, such as various types of displays, speakers, etc.; a storage unit 1008, such as a disk, an optical disk, etc.; and a communication unit 1009, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 1009 allows the device 1000 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0104] The computing unit 1001 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 1001 include, but are not limited to, a CPU (Central Processing Unit), a GPU (Graphic Processing Units), various dedicated AI (Artificial Intelligence) computing chips, various computing units running machine learning model algorithms, a DSP (Digital Signal Processor), and any appropriate processor, controller, microcontroller, etc. The computing unit 1001 performs the various methods and processes described above, such as the method of image acquisition. For example, in some embodiments, the method of image acquisition may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 1008. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 1000 via the ROM 1002 and / or the communication unit 1009. When the computer program is loaded into the RAM 1003 and executed by the computing unit 1001, one or more steps of the method described above may be performed. Alternatively, in other embodiments, the computing unit 1001 may be configured to execute the aforementioned image acquisition method in any other appropriate manner (for example, by means of firmware).

[0105] Various embodiments of the systems and techniques described above herein may be implemented in digital electronic circuit systems, integrated circuit systems, FPGAs (Field Programmable Gate Arrays), ASICs (Application-Specific Integrated Circuits), ASSPs (Application Specific Standard Products), SOCs (System On Chips), CPLDs (Complex Programmable Logic Devices), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include: being implemented in one or more computer programs that may be executed and / or interpreted on a programmable system including at least one programmable processor that may be a dedicated or general-purpose programmable processor that may receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0106] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0107] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any suitable combination of the foregoing. More specific examples of machine-readable storage media may include electrical connections based on one or more lines, portable computer disks, hard disks, RAM, ROM, EPROM (Electrically Programmable Read-Only-Memory) or flash memory, optical fiber, CD-ROM (Compact Disc Read-Only Memory), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0108] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (Cathode-Ray Tube) or LCD (Liquid Crystal Display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0109] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: LAN (Local Area Network), WAN (Wide Area Network), the Internet, and blockchain networks.

[0110] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services ("Virtual Private Server", or "VPS" for short). The server may also be a server of a distributed system, or a server combined with a blockchain.

[0111] It should be noted that artificial intelligence is a discipline that studies how computers can simulate certain human thought processes and intelligent behaviors (such as learning, reasoning, thinking, planning, etc.), and includes both hardware-level and software-level technologies. Artificial intelligence hardware technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, and big data processing; artificial intelligence software technologies mainly include computer vision technology, speech recognition technology, natural language processing technology, as well as machine learning / deep learning, big data processing technology, knowledge graph technology, and other major directions.

[0112] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0113] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. An image acquisition method, characterized in that: The method is applied to a vehicle, and comprises: receiving first data and second data, the first data comprising data collected by a motion sensor of the vehicle, and the second data comprising data acquired by a controller of the vehicle; Determine an adjustment angle of the camera module of the vehicle by using the first data and / or the second data, so as to adjust the angle of the camera module of the vehicle; adjusting the exposure time of the camera module of the vehicle to a first exposure time corresponding to the second data; A first image is captured by the camera module.

2. The method according to claim 1, characterized in that Determining the adjustment angle of the camera module of the vehicle by using the first data and / or the second data to adjust the angle of the camera module of the vehicle includes: Determine the movement amplitude and movement direction of the vehicle through the first data and / or the second data; Based on the movement amplitude and the movement direction, determining an adjustment angle of a camera module of the vehicle; Controlling a driver of the vehicle to send an adjustment instruction to an electric motor of the camera module, wherein the adjustment instruction includes the adjustment angle; Determining a moving direction corresponding to the adjustment instruction; Adjust the camera module to the moving direction.

3. The method according to claim 1, characterized in that The step of adjusting the exposure time of the camera module of the vehicle to the first exposure time corresponding to the second data includes: Determine a first corresponding relationship based on a linear relationship between the plurality of second data and the plurality of first exposure times; A first exposure time corresponding to the second data is determined according to the first corresponding relationship and the second data.

4. The method according to claim 1, characterized in that: The method further comprises: Preprocessing the first image to obtain a second image, wherein the preprocessing includes at least one of format conversion and image quality optimization; Processing the first data and the second data by a preset algorithm to obtain the motion state of the vehicle; According to the motion state, the field of view angle of the camera module is determined to adjust the display range of the second image.

5. The method according to any one of claims 1 to 4, characterized in that The camera module is one or more, and the method further includes: When there are multiple camera modules, the output images of the respective camera modules are stitched together to obtain a stitched image.

6. An image acquisition device, characterized in that: The device is configured in a vehicle, and comprises: A receiving module, configured to receive first data and second data, wherein the first data includes data collected by a motion sensor of the vehicle, and the second data includes data acquired by a controller of the vehicle; a first adjustment module, configured to determine an adjustment angle of the camera module of the vehicle according to the first data and / or the second data, so as to adjust an angle of the camera module of the vehicle; A second adjustment module, used for adjusting the exposure time of the camera module of the vehicle to a first exposure time corresponding to the second data; The acquisition module is used to acquire the first image through the camera module.

7. An image acquisition system, characterized in that: The image acquisition system is applied to a vehicle, and the system comprises: Camera module; The camera module includes one or more cameras and an image acquisition device, and the image acquisition device is used to adjust the angles of the one or more cameras based on first data collected by the vehicle's motion sensor and second data sent by the vehicle's controller, and adjust the first exposure time of the vehicle's camera module to a second exposure time corresponding to the second data.

8. The method according to claim 7, characterized in that The image acquisition device includes an electric motor and a driver, wherein the driver is connected to the electric motor and the motion sensor respectively. The driver is used to control the movement of the electric motor to adjust the angle of the camera module.

9. The method according to claim 8, characterized in that The image acquisition device is wirelessly connected to the controller of the vehicle for receiving second data.

10. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 5.

11. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-5.

12. A vehicle, characterized in that: It comprises the image acquisition system as described in any one of claims 7 to 9 or the electronic device as described in claim 10.

13. The vehicle according to claim 12, characterized in that Also includes: The motion sensor, wherein the motion sensor includes a gyroscope and an acceleration sensor, is used to collect the first data.

Citation Information

Cited By

  • Adjustment method and device of vehicle around view system, electronic equipment and storage medium

    CN119723490A