Video cropping method and apparatus for dashcam, device, and medium

Abstract

The present application relates to a video cropping method and apparatus for a dashcam, a device, and a medium. The video cropping method for a dashcam comprises: acquiring multi-channel video frames collected by a multi-channel dashcam, and acquiring cropping parameters corresponding to each video frame, wherein the cropping parameters comprise a cropping range; performing image cropping on each video frame according to the cropping range, so as to obtain cropped multi‑channel video frames; and performing encoding on the basis of the cropped multi‑channel video frames, and generating and displaying a target video. According to the technical solution of the present application, the display range of a dashcam video can be flexibly controlled.

Description

Video cropping method, device, equipment and medium for driving recorder

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to a Chinese patent application filed with the Patent Office of China on March 13, 2024, with application number 202410288449.6 and titled “Video cropping method, device, equipment and medium for driving recorder”, the entire contents of which are incorporated by reference into this application. Technical Field

[0003] The present application relates to the field of vehicle technology, and in particular to a method, device, equipment, and medium for cropping video of a driving recorder. Background Art

[0004] A driving recorder is a device installed on a car to record video and audio of the car's surroundings. The driving recorder has an audio and video playback function.

[0005] Currently, when playing back videos recorded by a dash cam, all recorded videos are transmitted to the vehicle computer for display. This display method is not flexible enough and the user experience is poor. Summary of the Invention

[0006] In order to solve the above technical problems, the present application provides a video cropping method, device, equipment and storage medium for a driving recorder.

[0007] In a first aspect, an embodiment of the present application provides a method for cropping video of a driving recorder, comprising:

[0008] Acquire multiple video frames captured by multiple driving recorders, and obtain cropping parameters corresponding to each video frame; wherein the cropping parameters include a cropping range;

[0009] Performing image cropping on each channel of video frames according to the cropping range to obtain cropped multi-channel video frames;

[0010] Encoding is performed based on the cropped multi-channel video frames to generate a target video and display the target video.

[0011] In a second aspect, an embodiment of the present application provides a video cropping device for a driving recorder, comprising:

[0012] An acquisition module, configured to acquire multiple video frames captured by multiple driving recorders, and to acquire cropping parameters corresponding to each video frame; wherein the cropping parameters include a cropping range;

[0013] A cropping module, configured to crop each channel of video frames according to the cropping range to obtain cropped multiple channels of video frames;

[0014] The display module is used to encode the cropped multi-channel video frames, generate a target video and display it.

[0015] In a third aspect, an embodiment of the present application provides an electronic device comprising: a processor; a memory for storing executable instructions of the processor; the processor for reading the executable instructions from the memory and executing the instructions to implement the video cropping method of the driving recorder described in the first aspect above.

[0016] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the storage medium stores a computer program, and when the computer program is executed by a processor, the video cropping method of the driving recorder described in the first aspect is implemented.

[0017] In a fifth aspect, an embodiment of the present application provides a vehicle, comprising the video cropping device of the driving recorder as described in the second aspect above, or the electronic device as described in the third aspect above.

[0018] The technical solution provided by the embodiment of the present application has the following advantages over the prior art: by obtaining multiple video frames captured by multiple driving recorders, and obtaining the cropping parameters corresponding to each video frame, each video frame is cropped according to the cropping parameters to obtain the cropped multiple video frames, encoding is performed based on the cropped multiple video frames, and the target video is generated and displayed. Therefore, the driving recorder video can be cropped according to the cropping parameters, the driving recorder video does not need to be displayed in full, and the display range of the driving recorder video can be flexibly controlled. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0020] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0021] FIG1 is a flow chart of a method for cropping a video of a driving recorder provided in an embodiment of the present application;

[0022] FIG2 is a flow chart of another method for cropping video of a driving recorder provided in an embodiment of the present application;

[0023] FIG3 is a schematic structural diagram of a video cropping device for a driving recorder provided in an embodiment of the present application. Specific embodiments

[0024] In order to more clearly understand the above-mentioned objectives, features and advantages of the present application, the scheme of the present application will be further described below. It should be noted that, in the absence of conflict, the embodiments of the present application and the features therein can be combined with each other.

[0025] In the following description, many specific details are set forth to facilitate a full understanding of the present application, but the present application can also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present application, not all of the embodiments.

[0026] Figure 1 is a flow chart of a video cropping method for a driving recorder provided in an embodiment of the present application. The method provided in an embodiment of the present application can be executed by a video cropping device of a driving recorder, which can be implemented using software and / or hardware and can be integrated into any electronic device with computing capabilities, such as a vehicle-mounted terminal.

[0027] As shown in FIG1 , the video cropping method of the driving recorder provided in the embodiment of the present application may include:

[0028] Step 101: obtain multiple video frames captured by multiple driving recorders, and obtain cropping parameters corresponding to each video frame.

[0029] In this embodiment, multiple driving recorders respectively perform video capture, and the vehicle-mounted device obtains multiple video frames captured by the multiple driving recorders. For example, five driving recorders respectively capture five video frames and transmit them to the vehicle-mounted device. The vehicle-mounted device obtains five video frames and obtains cropping parameters corresponding to the five video frames. The cropping parameters of each video frame can be the same or different. The five-channel driving recorder includes a main road and a four-channel surround-view driving recorder, wherein the main road is, for example, a PSD (Full Self-Drive, fully automatic driving) main road, and the four-channel surround view includes, for example, front, back, left, and right four channels, which are not limited here.

[0030] The cropping parameters include a cropping range. For example, the cropping parameters include a cropping start position and a cropping width and height. The cropping range is represented by the above two parameters.

[0031] The following describes how to obtain the cropping parameters.

[0032] As an example, obtaining the cropping parameters corresponding to each channel of video frames includes: receiving the cropping parameters corresponding to each channel of video frames input by a user.

[0033] Specifically, in the embodiments of the present application, the user refers to a developer, and the cropping parameters corresponding to each video frame can be set separately by, for example, a product developer. For example, a first cropping parameter is set for the main video frame, a second cropping parameter is set for the front video frame, a third cropping parameter is set for the rear video frame, a fourth cropping parameter is set for the left video frame, and a fifth cropping parameter is set for the right video frame. The first cropping parameter, the second cropping parameter, the third cropping parameter, the fourth cropping parameter, and the fifth cropping parameter can be the same or different, and the embodiments of the present application do not limit this.

[0034] As another example, obtaining the cropping parameters corresponding to each video frame includes: performing image recognition on each video frame to obtain a preset area in each video frame, and determining the cropping range corresponding to each video frame based on the preset area so that the cropped video frame contains the preset area, or does not contain the preset area.

[0035] For example, in an embodiment of the present application, the cropped video frame can be set to include a preset area or not include a preset area according to different vehicle models, different display effect requirements, etc. For example, if the vehicle model is large or a larger range of image effects needs to be displayed, the cropped video frame can be set to include a preset area. If the vehicle model is small or a smaller range of image effects needs to be displayed, and there is no need to display redundant images, the cropped video frame can be set to not include a preset area. This can be set by the developer according to actual needs.

[0036] In this example, a detection model is pre-trained to identify video frames through the detection model to obtain preset areas in the video frames, where the preset areas include door areas, license plate areas, etc. The detection model can be implemented based on machine learning or deep learning technology. For example, the detection model is a target detection model, which is implemented based on a target detection algorithm. The target detection model inputs an image and outputs a detection box of the target in the image. For another example, the detection model is an image semantic segmentation model, which is implemented based on an image semantic segmentation algorithm. The image semantic segmentation model inputs an image and outputs areas of each category in the image.

[0037] Step 102 : cropping each channel of video frames according to the cropping range to obtain cropped multiple channels of video frames.

[0038] In this embodiment, the vehicle-mounted device crops each video frame according to the cropping range of each video frame to obtain cropped video frames, wherein the cropped video frames conform to the cropping range. For example, the cropped video frames are cropped from the specified position of the original video frame image according to the cropping width and height.

[0039] As an example, a vehicle is equipped with an on-board computer, which includes an Android operating system and a QNX (Quick Unix) operating system. The QNX operating system is a Unix-like operating system for embedded systems. The Unix operating system is a multi-user, multi-tasking operating system, and the QNX operating system has higher performance than the Android operating system. Receiving cropping parameters corresponding to each video frame input by a user includes: receiving the cropping parameters transmitted by the Android operating system via the QNX operating system, wherein the cropping parameters are input by the user on the Android operating system side, and the cropping parameters are used to perform image cropping on the video frames via the QNX operating system. In this example, a multi-channel driving recorder captures multiple video frames via a camera. The on-board computer opens a camera interface via the Android operating system. When the user sets the cropping parameters, the user sets the cropping parameters via the Android operating system side of the on-board computer. The on-board computer then controls the Android operating system side to transmit the received cropping parameters to the QNX operating system side, and the QNX operating system performs the cropping operation. The relevant execution code for the cropping operation is pre-configured on the QNX operating system side, for example, an image processing algorithm such as C2D can be used.

[0040] Step 103 : Encode the cropped multi-channel video frames to generate and display a target video.

[0041] In this embodiment, for each of the cropped video frames, video encoding is performed on the cropped video frames to generate a video and display it on a designated display device. Video encoding is a method of converting a file in an original video format into a file in another video format using compression technology.

[0042] In one embodiment of the present application, the cropping parameters also include the video output resolution. Before encoding based on the cropped multi-channel video frames, generating the target video, and displaying it, the method further includes: obtaining the width and height of the cropped multi-channel video frames, and determining the scaling ratio of the cropped multi-channel video frames based on the width and height and the video output resolution, and scaling the cropped multi-channel video frames according to the scaling ratio. For example, if the cropping parameters include the output video width and height, which represent the video output resolution, the scaling ratio of the video frame is determined based on the ratio between the width and height of the cropped video frame and the output video width and height. Thus, the video can be displayed at different video resolutions to meet the demand for flexible scaling display of the cropped dashcam video.

[0043] According to the technical solution of the embodiment of the present application, by obtaining multiple video frames captured by multiple dash cams and obtaining the cropping parameters corresponding to each video frame, image cropping is performed on each video frame according to the cropping parameters to obtain cropped multiple video frames, encoding is performed based on the cropped multiple video frames, and the target video is generated and displayed. Thus, the dash cam video can be cropped according to the cropping parameters, the dash cam video does not need to be displayed in its entirety, and the display range of the dash cam video can be flexibly controlled.

[0044] Based on the above embodiment, FIG2 is a flow chart of another method for cropping a video of a driving recorder provided in an embodiment of the present application. As shown in FIG2 , the method includes:

[0045] Step 201 : obtaining multiple video frames collected by multiple driving recorders, performing image recognition on each video frame, and obtaining a preset area in each video frame.

[0046] In this embodiment, a detection model is pre-trained to identify video frames through the detection model to obtain preset areas in the video frames, wherein the preset areas include door areas, license plate areas, etc. The detection model can be implemented based on machine learning or deep learning technology. For example, the detection model is a target detection model, which is implemented based on a target detection algorithm. The target detection model inputs an image and outputs a detection box of the target in the image. For another example, the detection model is an image semantic segmentation model, which is implemented based on an image semantic segmentation algorithm. The image semantic segmentation model inputs an image and outputs areas of each category in the image.

[0047] The preset areas corresponding to the various video frames may be different. Optionally, performing image recognition on each video frame to obtain the preset areas in each video frame includes: determining the acquisition orientation of each video frame; performing image recognition on the video frames in the acquisition orientation; and obtaining the preset areas corresponding to the video frames in the acquisition orientation. That is, different preset area categories are set according to the acquisition orientation of the dashcam, and the acquisition orientation may include front, rear, left, and right roads. For example, performing image recognition on the left video frame obtains the left door area in the left video frame, and performing image recognition on the rear video frame obtains the license plate area in the rear video frame. Also, performing image recognition on the right video frame obtains the right door area in the right video frame, or performing image recognition on the front video frame obtains the road surface area in the front video frame, and so on.

[0048] Step 202 : determining a cropping range corresponding to each video frame according to a preset area, so that the cropped video frame includes the preset area, or does not include the preset area.

[0049] In this embodiment, according to the acquisition orientation of the driving recorder, it is determined whether the cropped video frame includes the preset area or does not include the preset area.

[0050] As an example, performing image recognition on each video frame to obtain a preset area in each video frame includes: performing image recognition on a side video frame to obtain a first area of ​​a first preset object in the side video frame, wherein the side video frame includes a left video frame and a right video frame. Determining a cropping range corresponding to each video frame based on the preset area includes: determining a cropping range corresponding to the side video frame based on the first area so that the cropped video frame includes the first area. In this example, the first preset object may be an object retained after cropping, and the first area of ​​the first preset object includes a vehicle door area.

[0051] As another example, image recognition is performed on each video frame to obtain a preset area in each video frame, including: image recognition is performed on the rear video frame to obtain a second area of ​​the second preset object in the rear video frame. The cropping range corresponding to each video frame is determined according to the preset area, including: determining the cropping range corresponding to the rear video frame according to the second area, so that the cropped video frame does not contain the second area. In this example, the second preset object may be an object that is removed after cropping, and the second area of ​​the second preset object includes the license plate area at the bottom of the picture. In addition, the second area may also include the top edge area at the top of the picture, or the top edge area on the left side of the picture, or the right area on the right side of the picture, etc. The second area may be any combination of one or more of the above-mentioned license plate area, top edge area, top edge area and right area. The embodiment of the present application may not limit the specific setting method of the second area.

[0052] Step 203 : cropping each channel of video frames according to the cropping range to obtain cropped multiple channels of video frames.

[0053] Step 204 : Encode the cropped multi-channel video frames to generate and display a target video.

[0054] In an embodiment of the present application, multiple video frames captured by multiple dash cams are obtained, image recognition is performed on each video frame, a preset area in each video frame is obtained, and a cropping range corresponding to each video frame is determined based on the preset area, so that the cropped video frame contains the preset area, or does not contain the preset area. Then, image cropping is performed on each video frame according to the cropping range to obtain cropped multiple video frames, encoding is performed based on the cropped multiple video frames, and a target video is generated and displayed. Thus, the dash cam video can be cropped according to the cropping parameters, the dash cam video may not be displayed in full, the display range of the dash cam video can be flexibly controlled, and the cropping scheme for the video captured by the dash cam in all directions can be automatically determined by using image recognition.

[0055] The embodiment of the present application also provides a video cropping device for a driving recorder.

[0056] FIG3 is a schematic structural diagram of a video cropping device for a driving recorder provided in an embodiment of the present application. As shown in FIG3 , the video cropping device for a driving recorder includes: an acquisition module 31 , a cropping module 32 , and a display module 33 .

[0057] The acquisition module 31 is configured to acquire multiple video frames collected by multiple driving recorders and to acquire cropping parameters corresponding to each video frame; wherein the cropping parameters include a cropping range;

[0058] A cropping module 32 is configured to crop each channel of video frames according to the cropping range to obtain cropped multiple channels of video frames;

[0059] The display module 33 is configured to perform encoding based on the cropped multi-channel video frames, generate a target video, and display the target video.

[0060] In one embodiment of the present application, the acquisition module 31 is specifically configured to:

[0061] Receive cropping parameters corresponding to each video frame input by the user; or,

[0062] Perform image recognition on each video frame to obtain a preset area in each video frame;

[0063] The cropping range corresponding to each video frame is determined according to the preset area, so that the cropped video frame includes the preset area, or does not include the preset area.

[0064] In one embodiment of the present application, the acquisition module 31 is specifically configured to:

[0065] Performing image recognition on the sidewalk video frame to obtain a first region of a first preset object in the sidewalk video frame;

[0066] A cropping range corresponding to the side video frame is determined according to the first area, so that the cropped video frame includes the first area.

[0067] In one embodiment of the present application, the acquisition module 31 is specifically configured to:

[0068] Performing image recognition on the back-channel video frame to obtain a second region of a second preset object in the back-channel video frame;

[0069] A cropping range corresponding to the subsequent video frame is determined according to the second region, so that the cropped video frame does not include the second region.

[0070] In one embodiment of the present application, the vehicle-mounted device includes an Android operating system and a QNX operating system, and the acquisition module 31 is specifically configured to:

[0071] The cropping parameters transmitted by the Android operating system are received through the QNX operating system, wherein the cropping parameters are input by a user on the Android operating system side, and the cropping parameters are used to perform image cropping on the video frame through the QNX operating system.

[0072] In one embodiment of the present application, the cropping parameters further include video output resolution, and the video cropping device of the driving recorder further includes:

[0073] a scaling module, configured to obtain the width and height of the cropped multi-channel video frames, and determine a scaling ratio of the cropped multi-channel video frames based on the width and height and the video output resolution;

[0074] The cropped multi-channel video frames are scaled according to the scaling ratio.

[0075] The video cropping device for a driving recorder provided in the embodiments of this application can execute any of the video cropping methods for a driving recorder provided in the embodiments of this application, and has the corresponding functional modules and beneficial effects. For details not fully described in the embodiments of the device of this application, please refer to the description of any of the method embodiments of this application.

[0076] An embodiment of the present application also provides an electronic device, which includes one or more processors and a memory.

[0077] The processor may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device to perform desired functions.

[0078] The memory may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory (cache), etc. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor may execute the program instructions to implement the methods of the embodiments of the present application described above and / or other desired functions. Various contents such as input signals, signal components, noise components, etc. may also be stored in the computer-readable storage medium.

[0079] In one example, the electronic device may further include an input device and an output device, these components being interconnected via a bus system and / or other forms of connection mechanisms. Furthermore, the input device may include, for example, a keyboard, a mouse, etc. The output device may output various information to the outside, including determined distance information, direction information, etc. The output device may include, for example, a display, a speaker, a printer, a communication network, and remote output devices connected thereto.

[0080] Of course, for simplicity, only some of the components in the electronic device related to the present application are shown, omitting components such as buses, input / output interfaces, etc. In addition, the electronic device may further include any other appropriate components depending on the specific application.

[0081] In addition to the above-mentioned methods and devices, the embodiments of the present application may also be a computer program product, which includes computer program instructions. When the computer program instructions are executed by a processor, the processor executes any method provided by the embodiments of the present application.

[0082] The computer program product may be written in any combination of one or more programming languages ​​to implement the program code for performing the operations of the embodiments of the present application, including object-oriented programming languages ​​such as Java, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0083] In addition, the embodiments of the present application may also be a computer-readable storage medium having computer program instructions stored thereon. When the computer program instructions are executed by a processor, the processor executes any method provided by the embodiments of the present application.

[0084] Computer readable storage media can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can, for example, include but is not limited to a system, device or component of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination thereof. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0085] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device comprising the element.

[0086] The foregoing description is intended only to provide specific embodiments of the present application, which will enable those skilled in the art to understand and implement the present application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application is not limited to the embodiments described herein, but is intended to be construed in the broadest manner consistent with the principles and novel features disclosed herein.

Claims

1. A method for cropping video from a driving recorder, comprising: Acquire multiple video frames captured by multiple driving recorders, and obtain cropping parameters corresponding to each video frame; wherein the cropping parameters include a cropping range; Performing image cropping on each channel of video frames according to the cropping range to obtain cropped multi-channel video frames; Encoding is performed based on the cropped multi-channel video frames to generate a target video and display the target video.

2. The method according to claim 1, wherein The obtaining of cropping parameters corresponding to each video frame includes: Receive the cropping parameters corresponding to each video frame input by the user.

3. The method according to claim 1, wherein The obtaining of cropping parameters corresponding to each video frame includes: Perform image recognition on each video frame to obtain a preset area in each video frame; The cropping range corresponding to each video frame is determined according to the preset area, so that the cropped video frame includes the preset area, or does not include the preset area.

4. The method according to claim 3, wherein: The performing image recognition on each video frame to obtain a preset area in each video frame includes: Determine the acquisition orientation of each video frame; Performing image recognition on the video frames of the acquisition orientation; The preset area corresponding to the video frame of the acquisition orientation is obtained.

5. The method according to claim 3 or 4, wherein: The performing image recognition on each video frame to obtain a preset area in each video frame includes: Performing image recognition on the sidewalk video frame to obtain a first region of a first preset object in the sidewalk video frame; The determining the cropping range corresponding to each video frame according to the preset area includes: A cropping range corresponding to the side video frame is determined according to the first area, so that the cropped video frame includes the first area.

6. The method according to claim 3 or 5, wherein: The performing image recognition on each video frame to obtain a preset area in each video frame includes: Performing image recognition on the back-channel video frame to obtain a second region of a second preset object in the back-channel video frame; The determining the cropping range corresponding to each video frame according to the preset area includes: A cropping range corresponding to the subsequent video frame is determined according to the second region, so that the cropped video frame does not include the second region.

7. The method of claim 2, wherein: The receiving of the cropping parameters corresponding to each video frame input by the user includes: The cropping parameters transmitted by the Android operating system are received through the QNX operating system, wherein the cropping parameters are input by a user on the Android operating system side, and the cropping parameters are used to perform image cropping on the video frame through the QNX operating system.

8. The method according to any one of claims 1 to 7, wherein: The cropping parameters also include video output resolution. Before encoding the cropped multi-channel video frames to generate and display the target video, the following steps are also included: Acquire the width and height of the cropped multi-channel video frames, and determine a scaling ratio of the cropped multi-channel video frames based on the width and height and the video output resolution; The cropped multi-channel video frames are scaled according to the scaling ratio.

9. A video cropping device for a driving recorder, wherein: include: An acquisition module, configured to acquire multiple video frames captured by multiple driving recorders, and to acquire cropping parameters corresponding to each video frame; wherein the cropping parameters include a cropping range; A cropping module, configured to crop each channel of video frames according to the cropping range to obtain cropped multiple channels of video frames; The display module is used to encode the cropped multi-channel video frames, generate a target video and display it.

10. An electronic device, wherein: include: processor; a memory for storing instructions executable by the processor; The processor is configured to read the executable instructions from the memory and execute the instructions to implement the video cropping method for a driving recorder as described in any one of claims 1 to 8.

11. A computer-readable storage medium, wherein: The storage medium stores a computer program, and when the computer program is executed by the processor, the video cropping method of the driving recorder described in any one of claims 1 to 8 is implemented.

12. A vehicle, wherein: include: The video cropping device of a driving recorder as claimed in claim 10, or the electronic device as claimed in claim 11.

Images