Multimedia information acquisition method and device, equipment, storage medium and vehicle

Abstract

The invention relates to a multimedia information acquisition method and device, equipment, a storage medium and a vehicle. The method comprises the following steps: in the driving process of a target vehicle, determining candidate auxiliary multimedia information of which the view finding direction is matched with the driving direction of the target vehicle in at least one piece of candidate auxiliary multimedia information as target auxiliary multimedia information; wherein the at least one piece of candidate auxiliary multimedia information is multimedia information generated by an advanced driving assistance system (ADAS) of the target vehicle; and performing image processing on the target auxiliary multimedia information to obtain chassis transparent multimedia information of the target vehicle. According to the embodiment of the invention, under the condition that the vehicle speed is relatively high, even if the vehicle passes by a longer distance in unit time, the ADAS can collect the multimedia information farther from the vehicle, and the corresponding chassis transparent multimedia information can still be generated according to the multimedia information collected by the ADAS; therefore, a driver can obtain the road condition at the bottom of the vehicle, and the driving safety of the vehicle is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of vehicle technology, and in particular to a method, device, equipment, storage medium and vehicle for acquiring multimedia information. Background Art

[0002] When the driver is driving the vehicle, he cannot know the road conditions under the vehicle chassis due to the obstruction of the vehicle chassis.

[0003] In the related art, the transparent chassis function in the Around View Monitor (AVM) system enables users to see the road surface under the vehicle. However, when the vehicle is at a high speed, the transparent chassis function cannot be used, and the driver cannot obtain the road surface conditions under the vehicle, which poses a threat to driving safety. Summary of the invention

[0004] In order to solve the above technical problems, the present disclosure provides a method, device, equipment, storage medium and vehicle for obtaining multimedia information.

[0005] In a first aspect, the present disclosure provides a method for acquiring multimedia information, the method comprising:

[0006] During the driving process of the target vehicle, determining, among at least one candidate auxiliary multimedia information, candidate auxiliary multimedia information whose viewing direction matches the driving direction of the target vehicle as target auxiliary multimedia information; wherein the at least one candidate auxiliary multimedia information is multimedia information generated by an advanced driving assistance system (ADAS) of the target vehicle;

[0007] The target auxiliary multimedia information is subjected to image processing to obtain chassis transparent multimedia information of the target vehicle.

[0008] In a second aspect, the present disclosure provides a multimedia information acquisition device, the device comprising:

[0009] A determination module, used for determining, during the driving process of the target vehicle, among at least one candidate auxiliary multimedia information, the candidate auxiliary multimedia information whose viewing direction matches the driving direction of the target vehicle as the target auxiliary multimedia information; wherein the at least one candidate auxiliary multimedia information is multimedia information generated by an advanced driving assistance system ADAS of the target vehicle;

[0010] The processing module is used to perform image processing on the target auxiliary multimedia information to obtain the chassis transparent multimedia information of the target vehicle.

[0011] In a third aspect, an embodiment of the present disclosure further provides an electronic device, including:

[0012] processor;

[0013] A memory for storing executable instructions;

[0014] The processor is used to read executable instructions from the memory and execute the executable instructions to implement the multimedia information acquisition method of the first aspect.

[0015] In a fourth aspect, an embodiment of the present disclosure further provides a computer-readable storage medium, wherein the storage medium stores a computer program. When the computer program is executed by a processor, the processor implements the multimedia information acquisition method of the first aspect.

[0016] In a fifth aspect, an embodiment of the present disclosure further provides a vehicle, comprising at least one of the following: the above-mentioned multimedia information acquisition device; the above-mentioned electronic device; and the above-mentioned computer-readable storage medium.

[0017] The technical solution provided by the embodiment of the present disclosure has the following advantages over the prior art: a multimedia information acquisition method, device, equipment and storage medium of the embodiment of the present disclosure, during the driving process of the target vehicle, the candidate auxiliary multimedia information whose framing direction matches the driving direction of the target vehicle in at least one candidate auxiliary multimedia information is determined as the target auxiliary multimedia information; wherein at least one candidate auxiliary multimedia information is multimedia information generated by the advanced driving assistance system ADAS of the target vehicle; image processing is performed on the target auxiliary multimedia information to obtain chassis transparent multimedia information of the target vehicle. With the above technical solution, since the multimedia information collected by ADAS can cover a range farther from the vehicle than the multimedia information collected by AVM, when the vehicle speed is high, even if the vehicle passes a longer distance per unit time, since ADAS can collect multimedia information farther from the vehicle, corresponding chassis transparent multimedia information can still be generated according to the multimedia information collected by the ADAS, thereby realizing the vehicle chassis transparent function in the high speed scene without adding additional hardware, so that the driver can obtain the road conditions at the bottom of the vehicle, and improve the driving safety of the vehicle. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0019] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0020] Figure 1 A flowchart of a method for acquiring multimedia information provided by an embodiment of the present disclosure;

[0021] Figure 2 A flowchart of another method for acquiring multimedia information provided by an embodiment of the present disclosure;

[0022] Figure 3 A flowchart of another method for acquiring multimedia information provided by an embodiment of the present disclosure;

[0023] Figure 4 A schematic diagram of calculating an offset distance provided in an embodiment of the present disclosure;

[0024] Figure 5 A block diagram of a target vehicle provided in an embodiment of the present disclosure;

[0025] Figure 6 A flowchart of another multimedia information acquisition method provided by an embodiment of the present disclosure;

[0026] Figure 7 A schematic diagram of a multimedia information sensor provided by an embodiment of the present disclosure when viewed from above;

[0027] Figure 8 A schematic diagram of an AVM camera provided in an embodiment of the present disclosure;

[0028] Fig. 9 A schematic diagram of an ADAS camera provided in an embodiment of the present disclosure;

[0029] Fig.10 A schematic diagram of the structure of a multimedia information acquisition device provided by an embodiment of the present disclosure;

[0030] Fig.11 A schematic diagram of the hardware circuit structure of a multimedia information acquisition device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0031] In order to more clearly understand the above-mentioned objectives, features and advantages of the present disclosure, the scheme of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features in the embodiments can be combined with each other without conflict.

[0032] In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present disclosure, rather than all of the embodiments.

[0033] When the driver is driving the vehicle, he cannot know the road conditions under the vehicle chassis due to the obstruction of the vehicle chassis.

[0034] In the related art, the transparent chassis function in the around view monitor (AVM) system enables the user to see the road surface under the vehicle. In the transparent chassis function, the image of the vehicle's forward direction collected by the AVM can be converted into an image of the vehicle chassis through image processing.

[0035] However, the image collected by AVM is close to the vehicle. When the vehicle speed is high, the distance covered by the vehicle in unit time is far. The single image collected by AVM cannot cover the complete chassis at the corresponding moment. It can only make the vehicle chassis partially transparent. The transparent chassis function cannot be used normally, and the driver cannot obtain the road conditions under the vehicle, posing a threat to driving safety.

[0036] In order to solve the above problems, the embodiments of the present disclosure provide a method, apparatus, device, storage medium and vehicle for obtaining multimedia information.

[0037] Figure 1 A flow chart of a multimedia information acquisition method provided for an embodiment of the present disclosure can be executed by a multimedia information acquisition device or a driving domain controller of a vehicle, wherein the device can be implemented using software and / or hardware and can generally be integrated into an electronic device, and the multimedia information acquisition device and / or electronic device can be configured in a first vehicle.

[0038] like Figure 1 As shown, the method includes:

[0039] Step 101, during the driving process of the target vehicle, determine the candidate auxiliary multimedia information whose viewing direction matches the driving direction of the target vehicle among at least one candidate auxiliary multimedia information as the target auxiliary multimedia information; wherein, at least one candidate auxiliary multimedia information is multimedia information generated by the advanced driving assistance system ADAS of the target vehicle.

[0040] The target vehicle may be a moving vehicle. This embodiment does not limit the energy supply type of the target vehicle. For example, the energy supply type of the target vehicle may be pure electric power or hybrid power. This embodiment does not limit the type of the target vehicle. For example, the type of the target vehicle may be a sports utility vehicle (SUV) or a sedan.

[0041] Advanced Driving Assistance System (ADAS), also known as advanced driving assistance system, can sense the surrounding environment during vehicle driving through sensors on the vehicle (for example, one or more of millimeter wave radar, laser radar, single / binocular camera), so as to realize auxiliary functions for the driver's driving. This embodiment does not limit the specific system included in the advanced driving assistance system. For example, the advanced driving assistance system may include a lane departure warning system (Lane Departure Warning System, LDWS) and / or a lane keeping system (Lane Keep Assistance, LKA). Multimedia information may include image information and / or video information, and the multimedia information may be a video frame included in the video. Candidate auxiliary multimedia information may be multimedia information collected by at least one multimedia information sensor in ADAS, and the candidate auxiliary multimedia information may correspond to a multimedia information sensor in ADAS one by one, that is, multimedia information collected by a multimedia information sensor in ADAS is a candidate auxiliary multimedia information. The framing direction may be the direction of the corresponding camera in ADAS when shooting the candidate auxiliary multimedia information. This embodiment does not impose any restriction on the framing direction. For example, the framing direction of the candidate auxiliary multimedia information photographed by the ADAS front-view camera may be the front, and the framing direction of the candidate auxiliary multimedia information photographed by the ADAS rear-view camera may be the rear.

[0042] In the disclosed embodiment, ADAS may have one or more cameras, each of which can generate corresponding candidate auxiliary multimedia information. During the driving process of the target vehicle, the orientation of each camera is used as the framing direction of the candidate auxiliary multimedia information generated by each camera. The driving direction of the target vehicle is determined by the gear mark of the target vehicle, the built-in navigation system of the target vehicle, etc., and the candidate auxiliary multimedia information whose framing direction matches the driving direction is determined as the target auxiliary multimedia information. Among them, the matching condition between the framing direction and the driving direction can be set according to user needs, etc., and this embodiment does not impose any restrictions. For example, the matching of the framing direction and the driving direction can be understood as the framing direction being consistent with the driving direction, or the framing direction includes the driving direction, etc.

[0043] In some embodiments of the present disclosure, among at least one candidate auxiliary multimedia information, candidate auxiliary multimedia information whose framing direction matches the driving direction of a target vehicle is determined as target auxiliary multimedia information, including: acquiring a gear position identification of the target vehicle; determining the driving direction of the target vehicle according to the gear position identification, and determining the candidate auxiliary multimedia information whose framing direction is consistent with the driving direction as the target auxiliary multimedia information.

[0044] Among them, the gear position identifier can be a variable that represents the gear position of the vehicle. The gear position identifier can correspond to the gear position one by one. For example, if the transmission type of the target vehicle is automatic gear, the gear position identifier can include: one or more of the D gear identifier, the S gear identifier, and the R gear identifier; the gear position identifier can also correspond to the gear direction one by one. For example, the gear position identifier can include: one or more of the forward gear identifier and the reverse gear identifier. The forward gear identifier can be the identifier corresponding to the gear position when the vehicle is moving forward. The reverse gear identifier can be the identifier corresponding to the gear position when the vehicle is moving backward.

[0045] The framing direction may correspond to the orientation of the camera one-to-one. For example, the framing direction corresponding to the front-facing camera facing forward may be the front, and the framing direction corresponding to the rear-facing camera facing backward may be the rear. Among them, the front-facing camera may be a camera for collecting multimedia information in front of the front of the vehicle. This embodiment does not limit the specific position of the front-facing camera. For example, the front-facing camera may be arranged on the roof or behind the windshield, and the height of the front-facing camera from the ground is higher than 1 meter. The rear-facing camera may be a camera for collecting multimedia information behind the vehicle parking space. This embodiment does not limit the specific position of the rear-facing camera. For example, the rear-facing camera may be arranged on the rear windshield or the high brake light, and the height of the rear-facing camera from the bottom surface is higher than 1 meter.

[0046] In the disclosed embodiment, the gear direction relationship between the gear mark and the driving direction can be preset, and the gear direction relationship can be set according to user needs, etc., and is not limited in the present embodiment. For example, the driving direction corresponding to the forward gear mark can be forward, and the driving direction corresponding to the reverse gear mark can be backward. The gear mark of the target vehicle can be determined according to the direction of the gear engaged by the target vehicle. For example, if the gear engaged by the target vehicle is the forward gear, the gear mark can be the forward gear mark; if the gear engaged by the target vehicle is the reverse gear, the gear mark can be the reverse gear mark. The multimedia information acquisition device acquires the gear mark of the target vehicle and determines whether the gear mark is a forward gear mark or a reverse gear mark.

[0047] If the gear position indicator is a forward gear indicator, the target vehicle's driving direction is determined to be forward according to the gear direction relationship, indicating that the target vehicle is driving forward, and the candidate auxiliary multimedia information generated by the ADAS front camera among the multiple candidate auxiliary multimedia information is determined as the target auxiliary multimedia information. If the gear position indicator is determined to be a reverse gear indicator, the target vehicle's driving direction is determined to be backward according to the gear direction relationship, indicating that the target vehicle is driving backward, and the candidate auxiliary multimedia information generated by the ADAS rear camera among the multiple candidate auxiliary multimedia information is determined as the target auxiliary multimedia information.

[0048] In the above scheme, the target auxiliary multimedia information is efficiently determined through the gear position identification, which creates a foundation for the subsequent realization of the chassis transparency function.

[0049] Step 102, performing image processing on the target auxiliary multimedia information to obtain chassis transparent multimedia information of the target vehicle.

[0050] Among them, the chassis transparent multimedia information can be the chassis part filled in the vehicle image, so that the user can know the multimedia information of the road condition under the vehicle chassis. Through the chassis transparent multimedia information, the visual blind spot at the bottom of the vehicle can be eliminated, achieving the visual effect of chassis transparency.

[0051] In the disclosed embodiment, after determining the target auxiliary multimedia information, the multimedia information acquisition device may perform image deformation, image cropping and other processing on all or part of the images included in the target auxiliary multimedia information, thereby obtaining the chassis transparent multimedia information of the target vehicle. If the target auxiliary multimedia information is a target auxiliary video, the image included in the target auxiliary multimedia information may be understood as the video frame included in the target auxiliary multimedia information.

[0052] In some embodiments of the present disclosure, image processing is performed on target auxiliary multimedia information to obtain transparent multimedia information of the chassis of the target vehicle, including: obtaining the turning angle information of the target vehicle, determining the cropping curvature according to the turning angle information, and determining the cropping length and cropping width according to the lens distortion rate corresponding to the target auxiliary multimedia information; performing image cropping processing on the target auxiliary multimedia information according to the cropping curvature, cropping length, and cropping width to obtain cropped multimedia information, and performing image deformation processing on the cropped multimedia information to obtain transparent multimedia information of the chassis.

[0053] The turning angle information may be information representing the turning angle of the vehicle. The clipping curvature may represent the curvature of the clipping frame for clipping the image in the direction of the vehicle's travel. The lens distortion rate may be a camera parameter of a camera in an ADAS, and the lens distortion rate may be related to the camera itself, and the lens distortion rate may represent the degree of perspective distortion of the lens. The clipping length may be the value of the clipping frame for clipping the image in the length direction of the vehicle body. The clipping width may be the value of the clipping frame for clipping the image in the width direction of the vehicle body.

[0054] In this embodiment, the multimedia information acquisition device can pre-acquire a standard interception frame of the target vehicle, and the standard interception frame can be related to one or more of the vehicle width, vehicle length, and vehicle speed of the target vehicle. During the driving process of the target vehicle, the multimedia information acquisition device can acquire the steering wheel angle information of the target vehicle, and convert the steering wheel angle information according to the corresponding relationship between the steering wheel rotation angle and the wheel rotation angle to obtain the turning angle information of the target vehicle wheel. According to the turning angle information, the curvature of the central axis of the updated interception frame corresponding to the standard interception frame is determined, and the curvature is the clipping curvature. In addition, the lens distortion rate of the camera parameters that generate the target auxiliary multimedia information is determined, and the length of the standard interception frame is adjusted according to the lens distortion rate to obtain the clipping length; the width of the standard interception frame is adjusted according to the distortion rate to obtain the clipping width.

[0055] The central axis of the standard frame is adjusted according to the clipping curvature to obtain the central axis of the updated clipping frame, and then the updated clipping frame can be determined when the central axis of the updated clipping frame, the clipping width, and the clipping length are determined. The image included in the target auxiliary multimedia information is clipped according to the updated clipping frame to obtain clipped multimedia information. The clipped multimedia information is subjected to image deformation processing to obtain corresponding chassis transparent multimedia information.

[0056] The multimedia information acquisition method provided by the embodiment of the present disclosure determines, during the driving process of the target vehicle, the target auxiliary multimedia information that matches the driving direction of the target vehicle in at least one candidate auxiliary multimedia information; wherein at least one candidate auxiliary multimedia information is multimedia information generated by the advanced driving assistance system ADAS of the target vehicle; image processing is performed on the target auxiliary multimedia information to obtain chassis transparent multimedia information of the target vehicle. In the embodiment of the present disclosure, since the multimedia information collected by ADAS can cover a range farther from the vehicle than the multimedia information collected by AVM, when the vehicle speed is high, even if the vehicle passes a longer distance per unit time, since ADAS can collect multimedia information farther from the vehicle, corresponding chassis transparent multimedia information can still be generated according to the multimedia information collected by the ADAS, thereby realizing the vehicle chassis transparent function in the high speed scenario without adding additional hardware, so that the driver can obtain the road conditions at the bottom of the vehicle, thereby improving the driving safety of the vehicle.

[0057] In addition, in the related art, the front camera of the AVM is usually arranged on the front bumper, and the height of the front camera of the AVM is about 0.3 meters from the ground. The rear camera of the AVM is usually arranged at the tailgate handle box, and the height of the rear camera of the AVM is about 0.7 meters from the ground. Since the arrangement height of the ADAS camera is higher than the arrangement height of the AVM camera, when the target auxiliary multimedia information collected by the ADAS is converted into two-dimensional chassis transparent multimedia information, the distortion of the image edge is small, and the edge blur is lighter. Moreover, compared with the multimedia information collected by the AVM, the target auxiliary multimedia information has more pixels, and the generated chassis transparent multimedia information has better clarity. Usually, the number of pixels of the target auxiliary multimedia information can be 5 million to 8 million.

[0058] Figure 2 A flowchart of another multimedia information acquisition method provided by an embodiment of the present disclosure, wherein the target auxiliary multimedia information is a target auxiliary video, such as Figure 2 As shown, before obtaining the chassis transparent multimedia information of the target vehicle, the method further includes:

[0059] Step 201, obtaining a surround view video generated by an AVM system of a target vehicle; wherein the surround view video is used to record the surrounding environment of the target vehicle.

[0060] The surround view video may be a video of the target vehicle's AVM that captures the vehicle's surrounding environment, and the surround view video may include one or more of the vehicle's front video, the vehicle's rear video, the vehicle's left side video, and the vehicle's right side video.

[0061] In this embodiment, the multimedia information acquisition device can acquire the panoramic AVM video generated by the AVM of the target vehicle, and retain the surround video recording the surrounding environment of the vehicle in the panoramic AVM video.

[0062] Step 202, determining multiple target surround view video frames displayed on a target display of a target vehicle among multiple candidate surround view video frames; wherein a first frame rate of the surround view video is greater than or equal to a display frame rate of the target display, and each candidate surround view video frame is a video frame included in the surround view video.

[0063] Among them, the target display can be a display device configured in the target vehicle. This embodiment does not limit the type of the target display. For example, the target display can be a vehicle-mounted screen or a vehicle-mounted projection. This embodiment does not limit the position of the target display. For example, the target display can be located in the center console between the main driver and the co-driver, and the target display can also be located in front of the main driver. The display frame rate can be the number of frames or images played per second by the target display. This embodiment does not limit the display frame rate. For example, the display frame rate can be 10 frames per second (Frame PerSecond, FPS). The first frame rate can be the number of frames or images included in the surround video per second. In order to display a smooth surround video on the target display, the first frame rate is greater than or equal to the display frame rate. This embodiment does not limit the first frame rate. For example, the first frame rate can be 30 frames per second. The candidate surround video frame can be a plurality of video frames included in the surround video. The target surround video frame can be a video frame displayed on the target display.

[0064] In this embodiment, the multimedia information acquisition device can obtain the first frame rate of the surround view video and the display frame rate of the target display, and filter the candidate surround view video frames included in the surround view video according to the first frame rate and the display frame rate to obtain the target surround view video frames displayed on the target display. For example, if the first frame rate and the display frame rate are the same, then the candidate surround view video frames included in the surround view video can all be target surround view video frames; if the first frame rate is 30 frames per second and the display frame rate is 10 frames per second, then the candidate surround view video frames every 2 candidate surround view video frames can be determined as target surround view video frames.

[0065] Step 203: determining matching video frames corresponding to each target surround video frame among a plurality of target auxiliary video frames included in the target auxiliary video.

[0066] The matching video frame may be a target auxiliary video frame corresponding to the target surround video frame. The road surface image included in the matching video frame may be located below the chassis of the target vehicle after the target vehicle has traveled a certain distance. Furthermore, part or all of the road surface image included in the matching video frame may be located below the chassis of the target vehicle at the time corresponding to the target surround video frame.

[0067] In this embodiment, the target surround video frames displayed on the target display are screened out, and for each target surround video frame, the matching video frame of the chassis transparent multimedia information at the corresponding moment of the target surround video frame is determined, thereby reducing the amount of subsequent image processing of the multimedia information and saving computing resources. The target surround video frame is matched with the matching video frame, thereby improving the adaptability of the transparent chassis subsequently generated based on the matching video and the target surround video frame, and optimizing the presentation effect of the transparent chassis.

[0068] It is understandable that after determining the matching video frame, in the subsequent process of performing image processing on the target auxiliary multimedia information to obtain the chassis transparent multimedia information, it is not necessary to perform graphic processing on all the target auxiliary multimedia information, but to perform image processing on the matching video frame.

[0069] Figure 3 A flowchart of another method for acquiring multimedia information provided by an embodiment of the present disclosure is shown as follows: Figure 3 As shown, determining the matching video frames corresponding to each target surround video frame among the multiple target auxiliary video frames included in the target auxiliary video includes:

[0070] Step 301, based on the synchronization matching frequency of ADAS and AVM, the display frame rate, the driving distance of the target vehicle, the average speed of the target vehicle, the target length of the target vehicle, the blind spot length of ADAS, and the second frame rate of the target auxiliary video, the target frame number difference corresponding to each target surround view video frame is calculated.

[0071] Among them, since the multimedia information of ADAS and the multimedia information of AVM are multimedia information generated by different systems, in order to make the multimedia information of ADAS and the multimedia information of AVM have good consistency, a synchronization matching frequency is provided in the present disclosure, and the synchronization matching frequency represents the frequency of frame synchronization of the multimedia information generated by ADAS and AVM. The synchronization matching frequency can be set according to the display frame rate, etc. For example, the synchronization matching frequency can be consistent with the display frame rate or less than the display frame rate. The driving distance of the target vehicle can be the distance passed by the target vehicle in one cycle. The average vehicle speed can be the average value of the driving speed of the target vehicle in one cycle. The target vehicle length can be the body length of the target vehicle. The blind spot length can be the length of the blind spot of the front-view camera or rear-view camera of the ADAS in the direction of vehicle travel. The second frame rate can be the number of frames or images included in the target auxiliary video per second. The target frame number difference can be the frame number difference between the target surround video frame and the target auxiliary video frame.

[0072] In this embodiment, the multimedia information acquisition device can determine the corresponding period according to the synchronization matching frequency, the display frame rate, etc., and determine the driving distance and the average speed of the vehicle in the period, determine the offset time of the target vehicle in the period according to the driving distance, the target vehicle length, the blind spot length, and the average speed, and determine the target frame number difference according to the offset time and the second frame rate.

[0073] In some embodiments of the present disclosure, the target frame number difference corresponding to each target surround view video frame is calculated based on the synchronization matching frequency of ADAS and AVM, the display frame rate, the driving distance of the target vehicle, the average speed of the target vehicle, the target length of the target vehicle, the blind spot length of ADAS, and the second frame rate of the target auxiliary video, including:

[0074] Step a1, based on multiple synchronization matching moments corresponding to the synchronization matching frequency and multiple display moments corresponding to the display frame rate, form multiple calculation cycles for calculating the target frame number difference; wherein, the first synchronization matching moment and the first display moment are the same moment, the starting moment of each calculation cycle is the synchronization matching moment, the ending moment of each calculation cycle is the display moment, and the first starting moment of the current calculation cycle is the synchronization matching moment closest to the ending moment of the previous calculation cycle.

[0075] Among them, the synchronization matching moment may be a time point at which ADAS and AVM perform frame synchronization determined according to the synchronization matching frequency. The display moment may be a time point at which the target display determines according to the display frame frequency to display an image. The first synchronization matching moment may be the first synchronization matching moment. The first display moment may be the first display moment. The calculation cycle may be a cycle for calculating the target frame number difference. The starting moment of the calculation cycle may be the starting time point of the calculation cycle. The ending moment of the calculation cycle may be the ending time point of the calculation cycle. The current calculation cycle may be a calculation cycle other than the first calculation cycle. The first starting moment is the starting moment of the current calculation cycle.

[0076] In the disclosed embodiment, the multimedia information acquisition device can determine multiple synchronization matching moments according to the synchronization matching frequency, and the time difference between two adjacent synchronization matching moments is the same. And determine multiple display moments according to the display frame rate, and the time difference between two adjacent display moments is the same. And determine multiple calculation cycles according to the synchronization matching moment and the display moment.

[0077] Specifically, the first synchronization matching moment and the first display moment can be set to the same moment, and the same moment can be the initial moment. The multimedia information acquisition device can determine the initial moment as the starting moment of the first calculation cycle, and determine the second display moment as the ending moment of the first calculation cycle. For other calculation cycles other than the first calculation cycle, the starting moment of the calculation cycle can be the synchronization matching moment closest to the ending moment of the previous calculation cycle, and the ending moment of the calculation cycle can be the first display moment after the ending moment of the previous calculation cycle.

[0078] Step a2, determine the current display cycle corresponding to the current calculation cycle; wherein the start time of the current display cycle is the display time closest to the start time of the current calculation cycle, and the end time of the current display cycle is the same as the end time of the current calculation cycle.

[0079] The start time of the current display cycle can be understood as the end time of the previous calculation cycle.

[0080] In this embodiment, the multimedia information acquisition device can determine the end time of the previous calculation cycle of the current calculation cycle as the start time of the current display cycle corresponding to the current calculation cycle, and determine the end time of the current calculation cycle as the end time of the current display cycle.

[0081] Step a3, calculate the offset distance of the target vehicle according to the driving distance of the target vehicle in the current display cycle, the target vehicle length, and the blind spot length, and calculate the initial offset duration of the current display cycle according to the offset distance and the average speed of the target vehicle in the current display cycle.

[0082] In this embodiment, the multimedia information acquisition device can calculate the number of wheel rotations of the target vehicle in the current display cycle, determine the travel distance of the target vehicle in the display according to the number of wheel rotations and the tire circumference, and calculate the sum of the target vehicle length and the blind spot length, and subtract half of the target vehicle length from the sum to obtain the offset distance of the target vehicle. For example, Figure 4 A schematic diagram of calculating an offset distance provided in an embodiment of the present disclosure is shown in FIG. Figure 4 As shown, if the driving distance in the current display cycle is S1, the target vehicle length is L, and the blind spot length is M, then the offset distance S2 = S1-L / 2+M.

[0083] After determining the offset distance, the offset distance is divided by the average speed of the target vehicle in the current display cycle to obtain the initial offset duration of the current display cycle. For example, if the offset distance is S2 and the average speed is v, the initial offset duration t = S2 / v.

[0084] Step a4, processing the initial offset duration according to the time relationship between the first starting time of the current display cycle and the second starting time of the current calculation cycle to obtain the target offset duration.

[0085] Among them, the second starting time can be the starting time of the current calculation cycle, and the time relationship between the first starting time and the second starting time can include: the first starting time is the same as the second starting time, the first starting time is before the second starting time, and the first starting time is after the second starting time.

[0086] In this embodiment, according to the time relationship between the first start time and the second start time, the initial offset duration is subjected to corresponding time period processing to obtain the target offset duration of the initial offset duration, wherein the time period processing includes one of maintaining processing, extending processing, and shortening processing.

[0087] In some embodiments of the present disclosure, the initial offset duration is processed according to the time relationship between the first starting moment of the current display cycle and the second starting moment of the current calculation cycle to obtain the target offset duration, including: if the first starting moment and the second starting moment are the same, the initial offset duration is determined as the target offset duration; if the first starting moment and the second starting moment are not the same, the time difference between the first starting moment and the second starting moment is determined, if the first starting moment is after the second starting moment, the sum of the initial offset duration and the time difference is used as the target offset duration, if the first starting moment is before the second starting moment, the difference between the initial offset duration and the time difference is used as the target offset duration.

[0088] In this embodiment, if the first starting moment and the second starting moment are the same, it means that the current calculation cycle and the current display cycle are the same, and the initial offset duration is used as the target offset duration. If the first starting moment and the second starting moment are different, the time difference between the first starting moment and the second starting moment is determined. If the first starting moment is after the second starting moment, it means that the current display cycle is shorter than the current calculation cycle. In order to calculate the target frame number difference based on the current calculation cycle, the initial offset duration and the time difference corresponding to the current display cycle are added to obtain the target offset duration. If the first actual moment is before the second starting moment, it means that the current display cycle is longer than the current calculation cycle. In order to calculate the target frame number difference based on the current calculation cycle, the initial offset duration and the time difference corresponding to the current display cycle are subtracted to obtain the target offset duration.

[0089] Step a5, calculating the target frame number difference of the current calculation cycle according to the target offset duration and the second frame rate.

[0090] In this embodiment, after determining the target offset duration, the target offset duration is divided by the inverse of the second frame rate to obtain the target frame number difference of the current calculation cycle. Specifically, if the target offset duration is T and the second frame rate is n, the target frame number difference N=Tn.

[0091] Step 302: Determine a reference video frame that matches each target surround view video frame in a target auxiliary video frame.

[0092] The reference video frame may be a video frame in the target auxiliary video frame that is closest in time to the target surround video frame.

[0093] In this embodiment, for each target surround video frame, the multimedia information acquisition device determines the surround time of the target surround video frame, and determines the target auxiliary video frame closest to the surround time among multiple target auxiliary video frames as the reference video frame.

[0094] Step 303 , determine the target auxiliary video frames in the target auxiliary video that are located before each reference video frame by a target frame number difference as matching video frames corresponding to each target surround video frame.

[0095] In this embodiment, after determining the target frame number difference and the reference video frame corresponding to the target surround video frame, the reference video frame is used as a reference and the target auxiliary video frame located at the target frame number difference before the reference video frame is used as the matching video frame of the target surround video frame.

[0096] In the above scheme, the determination of the matching video frame corresponding to the target surround video frame is realized, so that the chassis transparent multimedia information subsequently determined based on the target matching video frame is more adaptable to the surround video frame, and image processing of each frame in the target auxiliary multimedia information is avoided, thereby reducing the amount of invalid image processing and improving the effectiveness of image processing.

[0097] In some embodiments of the present disclosure, the chassis transparent multimedia information is a chassis transparent image. After obtaining the chassis transparent multimedia information of the target vehicle, the multimedia information acquisition method further includes: performing image stitching processing on each target surround video frame and the chassis transparent image corresponding to the target surround video frame to obtain a panoramic video frame of the target vehicle, and sending the panoramic video frame to a target display of the target vehicle.

[0098] The panoramic video frame may include images around the vehicle and the bottom of the vehicle.

[0099] In this embodiment, after determining the target surround video frame and the chassis transparent image corresponding to the target surround video frame, the multimedia information acquisition device can splice the target surround video frame and the chassis transparent image by using a bird's eye view (BEV) method, a bowl-shaped modeling method, etc., and perform smoothing processing such as interpolation processing on the spliced ​​portion to obtain a corresponding panoramic video frame. Further, the panoramic video frame is sent to a target display of the target vehicle to display the panoramic video frame to the user on the target display.

[0100] In the above scheme, a panoramic video of the vehicle is realized, so that the user can obtain the surrounding environment of the vehicle and the conditions of the chassis and road surface based on a video frame, thereby improving the user's information acquisition efficiency and improving the user's driving safety.

[0101] In some embodiments of the present disclosure, a bird's-eye view multimedia information sensor facing the road surface is installed at the bottom of the target vehicle. The multimedia information acquisition method also includes: when the speed of the target vehicle is less than or equal to a preset speed threshold, image processing is performed on the road surface multimedia information collected by the bird's-eye view multimedia information sensor to obtain transparent multimedia information of the chassis of the target vehicle.

[0102] The preset vehicle speed threshold can be set according to user needs, application scenarios, etc. For example, the preset vehicle speed threshold can be 0. The overhead multimedia information sensor can be a sensor for collecting road conditions on the chassis of the vehicle. This embodiment does not limit the type and model of the sensor. The road multimedia information can be a video, image, etc. obtained by the overhead multimedia information sensor.

[0103] In this embodiment, when the speed of the target vehicle is greater than a preset speed threshold, the target auxiliary multimedia information that matches the form direction of the target vehicle in at least one candidate auxiliary multimedia information of the ADAS is determined, and the chassis transparent multimedia information of the target vehicle is determined based on the target copy multimedia information. When the speed of the target vehicle is less than or equal to the preset speed threshold, the road multimedia information collected by the overhead multimedia information sensor is obtained, and the road multimedia information is processed by image cropping, image deformation, etc. to obtain the chassis transparent multimedia information of the target vehicle.

[0104] In this solution, since the target vehicle is stationary and has not moved, it is impossible to process the image of the road in front of the target vehicle and fill it under the vehicle after the vehicle moves. The chassis transparency function based on the multimedia information collected by ADAS is difficult to achieve. The road surface is directly photographed by the sensor facing the road surface in the chassis, which realizes the chassis transparency function when the vehicle is stationary, improving the driving safety of the vehicle when it is stationary.

[0105] Next, the multimedia information acquisition method in the embodiment of the present disclosure is further described through a specific example. Figure 5 A block diagram of a target vehicle provided by an embodiment of the present disclosure is shown in FIG. Figure 5As shown, the target vehicle includes: an ADAS front-view camera, an ADAS rear-view camera, a bottom camera (i.e., a top-view multimedia information sensor), a controller area network (CAN) bus, a driving domain controller, four surround-view cameras of the AVM, a display host, and a display screen (i.e., a target display). Among them, the wheel speed signal, the turning angle signal (i.e., the turning angle information), and the gear position signal (i.e., the gear position identification) of the wheel can be sent to the driving domain controller via the CAN bus, and the driving domain controller can send the video signal (i.e., the panoramic video frame) to the display host and the display screen.

[0106] Figure 6 A flowchart of another method for acquiring multimedia information provided by an embodiment of the present disclosure is shown as follows: Figure 6 As shown, the multimedia information acquisition method includes:

[0107] Step 601, obtain the speed of the target vehicle. If the speed is less than a preset speed threshold, turn on the overhead multimedia information sensor located under the target vehicle to collect road multimedia information, and generate a panoramic video based on the road multimedia information and the surround video collected by the AVM.

[0108] Specifically, the multimedia information acquisition device can read the wheel speed signal of the target vehicle. If the wheel speed signal is 0 kilometers per hour, the underbody camera is turned on to collect the underbody image and merge it with the AVM surround view camera. If it is not 0 kilometers per hour, the turning angle signal and gear signal are read. Figure 7 A schematic diagram of a multimedia information sensor provided by an embodiment of the present disclosure, such as Figure 7 As shown, the shaded part represents the image acquisition range of the overhead multimedia information sensor. It can be seen that the acquisition range of the overhead multimedia information sensor can cover the road surface at the bottom of the vehicle.

[0109] Step 602, if the gear position mark of the target vehicle is a forward gear mark, the multimedia information generated by the front-view camera of the ADAS is determined as the target auxiliary multimedia information; if the gear position mark of the target vehicle is a reverse gear mark, the multimedia information generated by the rear-view camera of the ADAS is determined as the target auxiliary multimedia information.

[0110] Specifically, if the gear position mark is the D gear mark, the front-view camera of the ADAS is called to collect target auxiliary multimedia information; if the gear position mark is the R gear mark, the rear-view camera of the ADAS is called to collect target auxiliary multimedia information.

[0111] Step 603: Determine the target surround view video frame displayed on the target display, and the matching video frame in the target auxiliary video corresponding to the target surround view video frame.

[0112] Specifically, the number of wheel rims that rotate per unit time can be determined based on the wheel speed signal, and the travel distance S1 of the target vehicle is the product of the number of wheel rims and the tire circumference. If the target vehicle length is L and the blind spot length is M, the offset distance S2 of the target vehicle is S1-L / 2+M. If the average speed per unit time is v, the initial offset duration t=S2 / v. If the second frame rate of the ADAS camera is n, and the target offset duration T is the same as the initial offset duration, the target frame difference N=Tn. Figure 8 A schematic diagram of an AVM camera provided in an embodiment of the present disclosure, Figure 8 The shaded area in the middle represents the image acquisition range of the AVM camera. Fig. 9 A schematic diagram of an ADAS camera provided in an embodiment of the present disclosure is shown in FIG. Fig. 9 The shaded area in the figure represents the image acquisition range of the ADAS camera. Figure 8 , Fig. 9 As shown in the figure, due to the high position of the ADAS camera, the blind area of ​​the ADAS camera is larger than that of the AVM camera. In some application scenarios, the AVM camera can capture the position very close to the front of the vehicle and the parking space, and the length of the blind area of ​​the AVM camera is approximately 0. In order to reduce the impact of the blind area on the generation of chassis transparent multimedia information, the impact of the blind area length M is considered when calculating the offset distance.

[0113] After determining the target frame number difference, the target auxiliary video frame N frames before the reference video frame of the target surround view video frame is determined as the matching video frame. It should be noted that there may be a frame synchronization signal between the ADAS and the AVM, through which the count can be re-calculated to eliminate the accumulated error.

[0114] Step 604 , cutting the matching video frame according to the rotation angle information and the lens distortion rate to obtain transparent multimedia information of the chassis of the target vehicle.

[0115] Specifically, the multimedia information acquisition device can determine the vehicle's turning angle information according to the steering wheel angle, calculate the vehicle's driving trajectory according to the turning angle information, determine the cropping curvature according to the driving trajectory, and determine the cropping length and cropping width according to the lens distortion rate.

[0116] Step 605 , stitching and smoothing the target surround video frame and the chassis transparent image to obtain a panoramic video frame.

[0117] Specifically, the cropped chassis transparent image is filled into the top view of the AVM, and the joints of the images are smoothed to obtain a panoramic video frame.

[0118] In the above scheme, the bottom vehicle transparent function can be realized even when the vehicle is not stationary and the speed is high. In addition, since the chassis transparent multimedia information is generated based on ADAS, the clarity of the bottom vehicle transparent multimedia information is improved compared with AVM, and the pixels are increased from 1 million to 3 million of AVM to 5 million to 8 million.

[0119] Fig.10 A schematic structural diagram of a multimedia information acquisition device provided by an embodiment of the present disclosure is shown.

[0120] In some embodiments of the present disclosure, Fig.10 The multimedia information acquisition device shown can be executed by an electronic device or a server. The electronic device may include but is not limited to mobile terminals such as smart phones, laptops, personal digital assistants (PDAs), tablet computers (PADs), portable multimedia information players (PMPs), vehicle terminals (such as vehicle navigation terminals), wearable devices, etc., and fixed terminals such as digital TVs, desktop computers, smart home devices, etc. The server can be a server cluster or a cloud server.

[0121] like Fig.10 As shown, the multimedia information acquisition device 1000 may include: a determination module 1001 and a processing module 1002 .

[0122] The determination module 1001 is used to determine, during the driving process of the target vehicle, candidate auxiliary multimedia information whose viewing direction matches the driving direction of the target vehicle among at least one candidate auxiliary multimedia information as target auxiliary multimedia information; wherein the at least one candidate auxiliary multimedia information is multimedia information generated by an advanced driving assistance system ADAS of the target vehicle;

[0123] The processing module 1002 is used to perform image processing on the target auxiliary multimedia information to obtain chassis transparent multimedia information of the target vehicle.

[0124] Optionally, the determining module 1001 is used to:

[0125] Obtaining a gear position identification of the target vehicle;

[0126] The driving direction of the target vehicle is determined according to the gear position identifier, and the candidate auxiliary multimedia information whose framing direction is consistent with the driving direction is determined as the target auxiliary multimedia information.

[0127] Optionally, the processing module 1002 is used to:

[0128] Obtaining turning angle information of the target vehicle;

[0129] Determine a trimming curvature according to the rotation angle information, and determine a trimming length and a trimming width according to a lens distortion rate corresponding to the target auxiliary multimedia information;

[0130] The target auxiliary multimedia information is subjected to image clipping processing according to the clipping curvature, the clipping length, and the clipping width to obtain clipping multimedia information, and the clipping multimedia information is subjected to image deformation processing to obtain the chassis transparent multimedia information.

[0131] Optionally, the target auxiliary multimedia information is a target auxiliary video, and the device further includes:

[0132] An acquisition module, used for acquiring a surround video generated by a panoramic monitoring imaging system AVM of the target vehicle before obtaining the chassis transparent multimedia information of the target vehicle; wherein the surround video is used to record the surrounding environment of the target vehicle;

[0133] The determination module is further used to determine a plurality of target surround view video frames displayed on a target display of the target vehicle among a plurality of candidate surround view video frames; wherein a first frame rate of the surround view video is greater than or equal to a display frame rate of the target display, and the plurality of candidate surround view video frames are video frames included in the surround view video;

[0134] The determination module is further used to determine matching video frames corresponding to each of the target surround video frames among a plurality of target auxiliary video frames included in the target auxiliary video.

[0135] Optionally, the determining module further includes:

[0136] a calculation unit, for calculating a target frame number difference corresponding to each of the target surround view video frames according to a synchronization matching frequency of the ADAS and the AVM, the display frame rate, a travel distance of the target vehicle, an average speed of the target vehicle, a target vehicle length of the target vehicle, a blind spot length of the ADAS, and a second frame rate of the target auxiliary video;

[0137] A determination unit, configured to determine, among target auxiliary video frames included in the target auxiliary video, reference video frames that are time-matched with each of the target surround view video frames;

[0138] The determination unit is further configured to determine the target auxiliary video frames in the target auxiliary video that are located before each of the reference video frames by the target frame number difference as matching video frames corresponding to each of the target surround video frames.

[0139] Optionally, the computing unit comprises:

[0140] A composition subunit, used to form a plurality of calculation cycles for calculating the target frame number difference according to a plurality of synchronization matching moments corresponding to the synchronization matching frequency and a plurality of display moments corresponding to the display frame rate; wherein the first synchronization matching moment and the first display moment are the same moment, the starting moment of each calculation cycle is the synchronization matching moment, the ending moment of each calculation cycle is the display moment, and the first starting moment of the current calculation cycle is the synchronization matching moment closest to the ending moment of the previous calculation cycle;

[0141] A determination subunit, used to determine a current display period corresponding to the current calculation period; wherein the start time of the current display period is the display time closest to the start time of the current calculation period, and the end time of the current display period is the same as the end time of the current calculation period;

[0142] A calculation subunit, configured to calculate an offset distance of the target vehicle according to a travel distance of the target vehicle in the current display cycle, the target vehicle length, and the blind spot length, and to calculate an initial offset duration of the current display cycle according to the offset distance and an average vehicle speed of the target vehicle in the current display cycle;

[0143] a processing subunit, configured to process the initial offset duration according to a time relationship between a first starting time of the current display cycle and a second starting time of the current calculation cycle to obtain a target offset duration;

[0144] The calculation subunit is further used to calculate the target frame number difference of the current calculation cycle according to the target offset duration and the second frame rate.

[0145] Optionally, the processing subunit is used to:

[0146] If the first starting time is the same as the second starting time, determining the initial offset duration as the target offset duration;

[0147] If the first starting time and the second starting time are not the same, the time difference between the first starting time and the second starting time is determined; if the first starting time is after the second starting time, the sum of the initial offset duration and the time difference is used as the target offset duration; if the first starting time is before the second starting time, the difference between the initial offset duration and the time difference is used as the target offset duration.

[0148] Optionally, the chassis transparent multimedia information is a chassis transparent image, and the device further includes:

[0149] The splicing module is used to, after obtaining the chassis transparent multimedia information of the target vehicle, perform image splicing processing on each of the target surround video frames and the chassis transparent image corresponding to the target surround video frame to obtain a panoramic video frame of the target vehicle, and send the panoramic video frame to the target display of the target vehicle.

[0150] Optionally, a multimedia information sensor facing the road surface is installed at the bottom of the target vehicle, and in the device:

[0151] The processing module is also used to perform image processing on the road multimedia information collected by the overhead multimedia information sensor to obtain transparent chassis multimedia information of the target vehicle when the speed of the target vehicle is less than or equal to a preset speed threshold.

[0152] The multimedia information acquisition device provided in the embodiments of the present disclosure can execute the multimedia information acquisition method provided in any embodiment of the present disclosure, and has the corresponding functional modules and beneficial effects of the execution method.

[0153] Fig.11 A schematic diagram of the hardware circuit structure of a multimedia information acquisition device provided by an embodiment of the present disclosure is shown.

[0154] like Fig.11 As shown, the multimedia information acquisition device 1100 may include a controller 1101 and a memory 1102 storing computer program instructions.

[0155] Specifically, the controller 1101 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0156] The memory 1102 may include a large capacity memory for information or instructions. By way of example and not limitation, the memory 1102 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a tape, or a universal serial bus (USB) drive or a combination of two or more of these. Where appropriate, the memory 1102 may include a removable or non-removable (or fixed) medium. Where appropriate, the memory 1102 may be inside or outside the integrated gateway device. In a specific embodiment, the memory 1102 is a non-volatile solid-state memory. In a specific embodiment, the memory 1102 includes a read-only memory (ROM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (Electrically Erasable Programmable ROM, EEPROM), an electrically alterable ROM (EAROM) or flash memory, or a combination of two or more of these.

[0157] The controller 1101 reads and executes the computer program instructions stored in the memory 1102 to perform the steps of the multimedia information acquisition method provided in the embodiment of the present disclosure.

[0158] In one example, the multimedia information acquisition device 1100 may further include a transceiver 1103 and a bus 1104. Fig.11 As shown, the controller 1101, the memory 1102 and the transceiver 1103 are connected via a bus 1104 and communicate with each other.

[0159] The bus 1104 includes hardware, software, or both. For example, but not limitation, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industrial Standard Architecture (ISA) bus, an InfiniBand interconnect, a Low Pin Count (LPC) bus, a Memory bus, a MicroChannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local Bus (VLB) bus, or other suitable buses or a combination of two or more of these. Where appropriate, the bus 1104 may include one or more buses. Although embodiments of the present application describe and illustrate a particular bus, the present application contemplates any suitable bus or interconnect.

[0160] The following is an embodiment of a computer-readable storage medium provided in an embodiment of the present disclosure. The computer-readable storage medium and the multimedia information acquisition method of the above-mentioned embodiments belong to the same inventive concept. For details not described in detail in the embodiment of the computer-readable storage medium, reference can be made to the embodiment of the above-mentioned lane keeping method.

[0161] This embodiment provides a storage medium including computer executable instructions. When the computer executable instructions are executed by a computer processor, they are used to execute a method for acquiring multimedia information.

[0162] Of course, the computer executable instructions of a storage medium containing computer executable instructions provided by an embodiment of the present disclosure are not limited to the above method operations, and can also execute related operations in the multimedia information acquisition method provided by any embodiment of the present disclosure.

[0163] Through the above description of the implementation methods, the technical personnel in the relevant field can clearly understand that the present disclosure can be implemented by means of software and necessary general hardware, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present disclosure can be essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ROM), random access memory (RAM), flash memory (FLASH), hard disk or optical disk, etc., including a number of instructions for a computer cloud platform (which can be a personal computer, a server, or a network cloud platform, etc.) to execute the multimedia information acquisition method provided by each embodiment of the present disclosure.

[0164] The embodiments of the present disclosure also provide a vehicle, comprising at least one of the following: the multimedia information acquisition device mentioned above; the electronic device mentioned above; and the computer-readable storage medium mentioned above.

[0165] Note that the above are only preferred embodiments of the present disclosure and the technical principles used. Those skilled in the art will understand that the present disclosure is not limited to the specific embodiments herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present disclosure. Therefore, although the present disclosure is described in more detail through the above embodiments, the present disclosure is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present disclosure, and the scope of the present disclosure is determined by the scope of the appended claims.

Claims

1. A method for acquiring multimedia information, It is characterized in that include: During the driving process of the target vehicle, determining, among at least one candidate auxiliary multimedia information, candidate auxiliary multimedia information whose viewing direction matches the driving direction of the target vehicle as target auxiliary multimedia information; wherein the at least one candidate auxiliary multimedia information is multimedia information generated by an advanced driving assistance system (ADAS) of the target vehicle; The target auxiliary multimedia information is subjected to image processing to obtain chassis transparent multimedia information of the target vehicle.

2. The method according to claim 1, It is characterized in that The step of determining, among at least one candidate auxiliary multimedia information, candidate auxiliary multimedia information whose viewing direction matches the driving direction of the target vehicle as target auxiliary multimedia information comprises: Obtaining a gear position identification of the target vehicle; The driving direction of the target vehicle is determined according to the gear position identifier, and the candidate auxiliary multimedia information whose framing direction is consistent with the driving direction is determined as the target auxiliary multimedia information.

3. The method according to claim 1, It is characterized in that The step of performing image processing on the target auxiliary multimedia information to obtain chassis transparent multimedia information of the target vehicle includes: Obtaining turning angle information of the target vehicle; Determine a trimming curvature according to the rotation angle information, and determine a trimming length and a trimming width according to a lens distortion rate corresponding to the target auxiliary multimedia information; The target auxiliary multimedia information is subjected to image clipping processing according to the clipping curvature, the clipping length, and the clipping width to obtain clipping multimedia information, and the clipping multimedia information is subjected to image deformation processing to obtain the chassis transparent multimedia information.

4. The method according to claim 1, It is characterized in that The target auxiliary multimedia information is a target auxiliary video. Before obtaining the chassis transparent multimedia information of the target vehicle, the method further includes: Acquire a surround view video generated by the panoramic monitoring imaging system AVM of the target vehicle; wherein the surround view video is used to record the surrounding environment of the target vehicle; Determine a plurality of target surround view video frames displayed on a target display of the target vehicle among a plurality of candidate surround view video frames; wherein a first frame rate of the surround view video is greater than or equal to a display frame rate of the target display, and each of the candidate surround view video frames is a video frame included in the surround view video; Determine a matching video frame corresponding to each of the target surround video frames among a plurality of target auxiliary video frames included in the target auxiliary video.

5. The method according to claim 4, It is characterized in that The determining of matching video frames corresponding to each of the target surround video frames among the multiple target auxiliary video frames included in the target auxiliary video includes: Calculate the target frame number difference corresponding to each of the target surround view video frames according to the synchronization matching frequency of the ADAS and the AVM, the display frame rate, the travel distance of the target vehicle, the average speed of the target vehicle, the target vehicle length of the target vehicle, the blind spot length of the ADAS, and the second frame rate of the target auxiliary video; Determine a reference video frame that matches each of the target surround view video frames in a target auxiliary video frame; The target auxiliary video frames in the target auxiliary video that are located before each of the reference video frames by the target frame number difference are determined as matching video frames corresponding to each of the target surround view video frames.

6. The method according to claim 5, It is characterized in that The target frame number difference corresponding to each target surround view video frame is calculated based on the synchronization matching frequency of the ADAS and the AVM, the display frame rate, the travel distance of the target vehicle, the average speed of the target vehicle, the target vehicle length of the target vehicle, the blind spot length of the ADAS, and the second frame rate of the target auxiliary video, including: According to the multiple synchronization matching moments corresponding to the synchronization matching frequency and the multiple display moments corresponding to the display frame rate, multiple calculation cycles for calculating the target frame number difference are formed; wherein the first synchronization matching moment and the first display moment are the same moment, the starting moment of each calculation cycle is the synchronization matching moment, the ending moment of each calculation cycle is the display moment, and the first starting moment of the current calculation cycle is the synchronization matching moment closest to the ending moment of the previous calculation cycle; Determine a current display period corresponding to the current calculation period; wherein the start time of the current display period is the display time closest to the start time of the current calculation period, and the end time of the current display period is the same as the end time of the current calculation period; Calculate the offset distance of the target vehicle according to the travel distance of the target vehicle in the current display cycle, the target vehicle length, and the blind spot length, and calculate the initial offset duration of the current display cycle according to the offset distance and the average speed of the target vehicle in the current display cycle; Processing the initial offset duration according to the time relationship between the first starting time of the current display cycle and the second starting time of the current calculation cycle to obtain a target offset duration; The target frame number difference of the current calculation period is calculated according to the target offset duration and the second frame rate.

7. The method according to claim 6, It is characterized in that The processing of the initial offset duration according to the time relationship between the first starting time of the current display cycle and the second starting time of the current calculation cycle to obtain the target offset duration includes: If the first starting time is the same as the second starting time, determining the initial offset duration as the target offset duration; If the first starting time and the second starting time are not the same, the time difference between the first starting time and the second starting time is determined; if the first starting time is after the second starting time, the sum of the initial offset duration and the time difference is used as the target offset duration; if the first starting time is before the second starting time, the difference between the initial offset duration and the time difference is used as the target offset duration.

8. The method according to claim 4, It is characterized in that The chassis transparent multimedia information is a chassis transparent image. After obtaining the chassis transparent multimedia information of the target vehicle, the method further includes: Each of the target surround view video frames and the chassis transparent image corresponding to the target surround view video frame is subjected to image stitching processing to obtain a panoramic video frame of the target vehicle, and the panoramic video frame is sent to a target display of the target vehicle.

9. The method according to claim 1, It is characterized in that A multimedia information sensor facing the road surface is installed at the bottom of the target vehicle, and the method further includes: When the speed of the target vehicle is less than or equal to a preset speed threshold, image processing is performed on the road multimedia information collected by the overhead multimedia information sensor to obtain transparent chassis multimedia information of the target vehicle.

10. A multimedia information acquisition device, It is characterized in that include: A determination module, used for determining, during the driving process of the target vehicle, among at least one candidate auxiliary multimedia information, the candidate auxiliary multimedia information whose viewing direction matches the driving direction of the target vehicle as the target auxiliary multimedia information; wherein the at least one candidate auxiliary multimedia information is multimedia information generated by an advanced driving assistance system ADAS of the target vehicle; The processing module is used to perform image processing on the target auxiliary multimedia information to obtain the chassis transparent multimedia information of the target vehicle.

11. An electronic device, It is characterized in that The electronic device comprises: Processor and memory; The processor is used to execute the steps of the method according to any one of claims 1 to 9 by calling the program or instruction stored in the memory.

12. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores a program or instruction, and the program or instruction enables a computer to execute the steps of the method according to any one of claims 1 to 9.

13. A vehicle, It is characterized in that Include at least one of the following: The multimedia information acquisition device described in claim 10; The electronic device as claimed in claim 11; The computer readable storage medium of claim 12.

Images