Video stream processing method and device, vehicle and storage medium

Abstract

The invention provides a video stream processing method and device, a vehicle and a storage medium, and relates to the technical field of automatic driving. The method comprises the following steps: encoding each initial video frame in a video stream by adopting a video encoder to obtain an encoded video frame corresponding to each initial video frame; determining a target frame number corresponding to each service type according to the sending frame rate corresponding to each service type and the initial frame rate corresponding to the video stream; based on the target frame number corresponding to each service type, determining a sending mode of sending the coded video frame to a video stream receiving module corresponding to each service type; and sending the coded video frame to a video stream receiving module corresponding to each service type based on the sending mode corresponding to each service type. Therefore, the resource usage amount of the video encoder can be reduced, and video recording requirements of different service types can be met.

Description

Technical Field

[0001] The present disclosure relates to the field of autonomous driving technology, and in particular to a video stream processing method, device, vehicle, and storage medium. Background Art

[0002] Video data recorded by vehicle cameras can be applied to the video recording needs of different businesses. For example, two rearview cameras need to support both shadow mode video recording and the video recording needs of the digital video recorder (DVR) in the smart cockpit. However, these two needs have different frame rate requirements for the cameras. The DVR requires a higher frame rate to ensure real-time data transmission to the cockpit screen. Shadow mode is mainly used to verify algorithm problems during operation, requiring data to be transmitted back to the cloud, and has a lower frame rate requirement. Summary of the Invention

[0003] The present disclosure aims to solve one of the technical problems in the related art at least to a certain extent.

[0004] The first embodiment of the present disclosure provides a video stream processing method, including:

[0005] Encoding each initial video frame in the video stream using a video encoder to obtain an encoded video frame corresponding to each initial video frame;

[0006] Determining a target number of frames corresponding to each service type according to a sending frame rate corresponding to each service type and an initial frame rate corresponding to the video stream;

[0007] Determining, based on a target number of frames corresponding to each of the service types, a sending mode for sending the encoded video frame to a video stream receiving module corresponding to each of the service types;

[0008] Based on the sending mode corresponding to each of the service types, the encoded video frame is sent to the video stream receiving module corresponding to each of the service types.

[0009] A second embodiment of the present disclosure provides a video stream processing device, including:

[0010] An encoding module, configured to encode each initial video frame in the video stream using a video encoder to obtain an encoded video frame corresponding to each initial video frame;

[0011] A first determining module is configured to determine a target number of frames corresponding to each service type according to a sending frame rate corresponding to each service type and an initial frame rate corresponding to the video stream;

[0012] A second determining module is configured to determine a sending mode for sending the encoded video frame to a video stream receiving module corresponding to each of the service types based on a target number of frames corresponding to each of the service types;

[0013] The sending module is used to send the encoded video frame to the video stream receiving module corresponding to each of the service types based on the sending mode corresponding to each of the service types.

[0014] The third aspect embodiment of the present disclosure proposes a vehicle, comprising: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to: perform the steps of the video stream processing method proposed in the first aspect embodiment of the present disclosure.

[0015] The fourth embodiment of the present disclosure proposes a computer-readable storage medium. When the instructions in the storage medium are executed by the processor of the mobile terminal, the mobile terminal can perform the steps of the video stream processing method proposed in the first embodiment of the present disclosure.

[0016] The video stream processing method, device, vehicle, and storage medium provided by the present disclosure have the following beneficial effects:

[0017] In the embodiment of the present disclosure, a video encoder is used to encode each initial video frame in a video stream to obtain an encoded video frame corresponding to each initial video frame, and then the target number of frames corresponding to each business type is determined based on the sending frame rate corresponding to each business type and the initial frame rate corresponding to the video stream. Based on the target number of frames corresponding to each business type, the sending method of sending the encoded video frame to the video stream receiving module corresponding to each business type is determined, and finally, based on the sending method corresponding to each business type, the encoded video frame is sent to the video stream receiving module corresponding to each business type. Thus, after each video frame is encoded by a video encoder, the corresponding sending method is determined according to the video recording requirements of each business type, and then the encoded video frame is sent to the video stream receiving module corresponding to each business type, thereby not only reducing the resource usage of the video encoder, but also adapting to the video recording requirements of different business types.

[0018] Additional aspects and advantages of the present disclosure will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The above and / or additional aspects and advantages of the present disclosure will become apparent and readily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0020] Figure 1 A flowchart of a video stream processing method provided by an embodiment of the present disclosure;

[0021] Figure 2 A schematic flow chart of a video stream processing method provided by another embodiment of the present disclosure;

[0022] Figure 3 A schematic structural diagram of a video stream processing device provided by another embodiment of the present disclosure;

[0023] Figure 4 A block diagram of an exemplary electronic device suitable for implementing embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0024] The following describes in detail embodiments of the present disclosure, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present disclosure, and should not be construed as limiting the present disclosure.

[0025] The video stream processing method, device, vehicle, and storage medium according to the embodiments of the present disclosure are described below with reference to the accompanying drawings.

[0026] The embodiment of the present disclosure takes the video stream processing method as an example in which it is configured in a video stream processing device. The video stream processing device can be applied to any vehicle so that the vehicle can perform the video stream processing function.

[0027] Figure 1 A flowchart of a video stream processing method provided by an embodiment of the present disclosure is provided.

[0028] like Figure 1 As shown, the video stream processing method may include the following steps:

[0029] Step 101: Encode each initial video frame in a video stream using a video encoder to obtain an encoded video frame corresponding to each initial video frame.

[0030] In some embodiments, each initial video frame may be encoded using uniform encoding parameters.

[0031] In some embodiments, some initial video frames in a video stream may also be encoded as key frames. A key frame is a special type of frame, also known as an I-frame (Intra Frame). A key frame is an important frame in a video sequence that contains complete image information and can be decoded independently of other frames.

[0032] Since the key frame contains complete image information, it can ensure the quality of the video and avoid the degradation of video quality due to decoding errors of other frames. However, if every frame is modified into a key frame, the video stream will be very large.

[0033] Therefore, in the embodiment of the present disclosure, the initial video frame corresponding to the preset driving scene can be encoded based on the encoding parameters corresponding to the key frame to obtain the encoded video frame, so that the status around the vehicle can be clearly recorded when the vehicle is in an important scene.

[0034] Among them, the preset driving scenario may include at least one of the following: the vehicle is turning, the distance between the vehicle and the obstacle is less than a distance threshold, the vehicle collides, etc.

[0035] The encoding parameters corresponding to the key frames are parameters used to encode the initial video frames into the key frames.

[0036] Optionally, when the video encoder receives a key frame generation message, a preset number of initial video frames obtained after receiving the key frame generation message are determined as initial video frames corresponding to a preset driving scenario.

[0037] The key frame generation message is triggered when the vehicle is in a preset driving scenario. For example, when the vehicle turns, a key frame generation message is triggered; when the distance between the vehicle and an obstacle is detected to be less than a distance threshold, a key frame generation message is triggered.

[0038] In some embodiments, after a key frame generation message is triggered, the message may be sent to a video encoder via a message middleware. After receiving the key frame generation message, the video encoder encodes a preset number of initial video frames received thereafter as key frames.

[0039] It's important to note that if the camera's frame rate is 30 Hz, keyframe generation should take less than 30 milliseconds. Any delay exceeding 30 milliseconds will affect the camera's normal frame rate, causing frame drops and degrading image quality. The delay from triggering keyframe generation to actual generation must be less than 50 milliseconds, otherwise critical information may be missed.

[0040] The preset number may be 1 frame or 2 frames, which is not limited in the present disclosure.

[0041] In some embodiments, a target video frame in a video stream is determined based on a preset frame interval, and the target video frame is encoded based on encoding parameters to obtain an encoded video frame.

[0042] The preset frame interval may be 19 frames, 20 frames, etc., which is not limited in the present disclosure.

[0043] In some embodiments, if the 9th and 15th video frames in the video stream are preset driving scenes and the frame interval is 19 frames, the 1st, 9th, 15th, 20th, and 24th video frames in the video stream are encoded as key frames, and the other video frames are encoded as ordinary frames.

[0044] In some embodiments, after obtaining the encoded video frame corresponding to the initial video frame, the encoded video frame may be cached using a cache module.

[0045] Step 102: Determine the target number of frames corresponding to each service type according to the sending frame rate corresponding to each service type and the initial frame rate corresponding to the video stream.

[0046] Among them, the service type can be driving recorder video recording, shadow mode video recording, etc., which is not limited in this disclosure.

[0047] Among them, shadow mode can be understood as: when someone is driving, the system including sensors is still running but does not participate in vehicle control. It only verifies the decision-making algorithm. That is, the system algorithm makes continuous simulated decisions in "shadow mode" and compares the decisions with the driver's behavior. Once the two are inconsistent, the scenario is judged as an "extreme working condition", which triggers data feedback.

[0048] The initial frame rate is the acquisition frame rate of the camera, for example, the initial frame rate may be 30 Hz, or 30 frames per second. The present disclosure does not limit the initial frame rate.

[0049] The sending frame rate may be the number of times per second that the video stream is sent to the video stream receiving module corresponding to the service type. For example, if the sending frame rate corresponding to the service type is 30 Hz, the video stream is sent to the video stream receiving module corresponding to the service type 30 times per second.

[0050] The target frame rate can be determined as the ratio of the initial frame rate to the transmitted frame rate. For example, if the initial frame rate is 10 Hz and the transmitted frame rate corresponding to shadow mode video recording is 2 Hz, the target frame rate corresponding to shadow mode video recording can be 15 frames.

[0051] Step 103 : Based on the target number of frames corresponding to each service type, a sending method for sending the encoded video frames to the video stream receiving module corresponding to each service type is determined.

[0052] In some embodiments, in response to the first service type corresponding to a plurality of target frames, the sending mode is determined to compress the target number of encoded video frames into a video segment and send it to the video stream receiving module corresponding to the first service type.

[0053] For example, if the first service type is shadow mode video recording and the target frame number is 15, then 15 encoded video frames are compressed into a video segment and the compressed video segment is sent to the video stream receiving module corresponding to shadow mode video recording. That is, every time 15 encoded video frames are buffered, the 15 frames are compressed into a video segment and sent to the video stream receiving module corresponding to shadow mode video recording.

[0054] In some embodiments, there may be one or more first service types, and the target number of frames corresponding to each first service type may be the same or different. Whenever the number of cached encoded video frames reaches the target number of frames corresponding to any first service type, the encoded video frames of the target number of frames are compressed into video segments and sent to the video stream receiving module corresponding to any first service type.

[0055] In some embodiments, in response to the target number of frames corresponding to the second service type being one, the sending mode is determined to be to send each encoded video frame directly to the video stream receiving module corresponding to the second service type.

[0056] For example, if the second service type is dashcam video recording and the target frame number is 1 frame, then compression is not required and 1 frame of encoded video frame is directly sent to the dashcam.

[0057] In some embodiments, there may be one or more second service types, which is not limited in this disclosure.

[0058] Step 104 : Based on the sending mode corresponding to each service type, the encoded video frame is sent to the video stream receiving module corresponding to each service type.

[0059] In the embodiment of the present disclosure, a video encoder is used to encode each initial video frame in a video stream to obtain an encoded video frame corresponding to each initial video frame, and then the target number of frames corresponding to each business type is determined based on the sending frame rate corresponding to each business type and the initial frame rate corresponding to the video stream. Based on the target number of frames corresponding to each business type, the sending method of sending the encoded video frame to the video stream receiving module corresponding to each business type is determined, and finally, based on the sending method corresponding to each business type, the encoded video frame is sent to the video stream receiving module corresponding to each business type. Thus, after each video frame is encoded by a video encoder, the corresponding sending method is determined according to the video recording requirements of each business type, and then the encoded video frame is sent to the video stream receiving module corresponding to each business type, thereby not only reducing the resource usage of the video encoder, but also adapting to the video recording requirements of different business types.

[0060] Figure 2 A flowchart of a video stream processing method provided by an embodiment of the present disclosure is provided.

[0061] like Figure 2 As shown, the video stream processing method may include the following steps:

[0062] Step 201: After the image acquisition device in the vehicle is started, a first internal reference is read from the image acquisition device.

[0063] Among them, the image acquisition device can be a camera, a webcam, etc., which is not limited in this disclosure.

[0064] It should be noted that in the field of autonomous driving, internal references are needed to calibrate captured images so that the autonomous driving system can better understand and perceive the surrounding environment and make more accurate decisions. The accuracy of internal references is crucial to the safety and reliability of autonomous driving, so extracting accurate and reliable internal references is a very important part of the autonomous driving system.

[0065] The internal parameters are burned into the camera module when the camera leaves the factory. Each camera is different. Therefore, the most reliable method is to read the internal parameters from the camera module every time you use the camera. This can ensure that the internal parameters you are using correspond to the camera.

[0066] However, internal parameters are obtained through the I2C interface and are read from the camera module when the camera is turned on. However, after the autonomous driving control domain is turned on, the camera begins to collect video data and transmit data via I2C. If the internal parameters are read at the same time, it will cause an I2C conflict and cause the camera to malfunction.

[0067] Therefore, in the embodiment of the present disclosure, after the image acquisition device is started, the internal reference is obtained first, and then the video data is acquired, so as to avoid conflicts.

[0068] In some embodiments, when obtaining, the first internal parameter is compared with the pre-stored second internal parameter, and if the first internal parameter is different from the pre-stored second internal parameter, the pre-stored second internal parameter is updated to the first internal parameter.

[0069] In the embodiment of the present disclosure, the stored second internal reference can be used to process the collected data using the second internal reference when the internal reference is not read after the image acquisition device is started.

[0070] In the disclosed embodiment, internal parameters are read once each time the autonomous driving control domain is activated. This ensures that even if a camera is replaced, the internal parameters obtained are consistent with the actual connected camera. The internal parameter results are also saved, preventing the use of the last read internal parameter in the event of an internal parameter reading failure, making internal parameter acquisition more robust.

[0071] Step 202: Control the image acquisition device to acquire original video data.

[0072] Step 203: Process the original video data based on the first internal reference to obtain a video stream.

[0073] Specifically, the original video data may be converted from the camera coordinate system to the image coordinate system based on the first internal parameter.

[0074] Step 204 : Encode each initial video frame in the video stream using a video encoder to obtain an encoded video frame corresponding to each initial video frame.

[0075] Step 205 : Determine the target number of frames corresponding to each service type according to the sending frame rate corresponding to each service type and the initial frame rate corresponding to the video stream.

[0076] Step 206 : Based on the target number of frames corresponding to each service type, determine a sending method for sending the encoded video frames to the video stream receiving module corresponding to each service type.

[0077] Step 207: Based on the sending mode corresponding to each service type, the encoded video frame is sent to the video stream receiving module corresponding to each service type.

[0078] The specific implementation of steps 204 to 207 can refer to the detailed description in other embodiments of the present disclosure and will not be described in detail here.

[0079] In the embodiment of the present disclosure, after the image device is started, the first internal reference is read from the image acquisition device, and then the image acquisition device is controlled to acquire raw video data, and the raw video data is processed based on the first internal reference to obtain a video stream, and then each initial video frame in the video stream is encoded using a video encoder to obtain an encoded video frame corresponding to each initial video frame, and then the target number of frames corresponding to each business type is determined based on the sending frame rate corresponding to each business type and the initial frame rate corresponding to the video stream, and based on the target number of frames corresponding to each business type, the sending mode of sending the encoded video frame to the video stream receiving module corresponding to each business type is determined, and finally, based on the sending mode corresponding to each business type, the encoded video frame is sent to the video stream receiving module corresponding to each business type. Thus, after the image acquisition device is started, the first internal reference is read first, and then the image acquisition device is controlled to acquire video data, thereby avoiding conflicts caused by using I2C to read the first internal reference and the raw video data at the same time, ensuring the accuracy of the acquired raw video data and the first internal reference, and thus ensuring the accuracy of the video stream to be encoded, providing accurate video data for different business types.

[0080] In order to implement the above embodiments, the present disclosure also proposes a video stream processing device.

[0081] Figure 3 This is a structural diagram of a video stream processing device provided by an embodiment of the present disclosure.

[0082] like Figure 3 As shown, the video stream processing device 300 may include:

[0083] The encoding module 301 is configured to encode each initial video frame in the video stream using a video encoder to obtain an encoded video frame corresponding to each initial video frame;

[0084] A first determining module 302 is configured to determine a target number of frames corresponding to each service type based on a sending frame rate corresponding to each service type and an initial frame rate corresponding to a video stream;

[0085] A second determining module 303 is configured to determine a sending mode for sending the encoded video frames to the video stream receiving module corresponding to each service type based on the target number of frames corresponding to each service type;

[0086] The sending module 304 is configured to send the encoded video frame to the video stream receiving module corresponding to each service type based on the sending mode corresponding to each service type.

[0087] In some possible implementations, the second determining module 303 is configured to:

[0088] In response to the number of target frames corresponding to the first service type being multiple, determining a sending mode to compress the coded video frames of the target number of frames into one video segment and sending it to a video stream receiving module corresponding to the first service type; and / or,

[0089] In response to the target number of frames corresponding to the second service type being one, the sending mode is determined to be to directly send each encoded video frame to the video stream receiving module corresponding to the second service type.

[0090] In some possible implementations, the encoding module is configured to:

[0091] Encoding the initial video frame corresponding to the preset driving scene based on the encoding parameters corresponding to the key frame to obtain an encoded video frame; and / or,

[0092] Based on a preset frame interval, a target video frame in a video stream is determined, and based on encoding parameters, the target video frame is encoded to obtain an encoded video frame.

[0093] In some possible implementations, the encoding module is further configured to:

[0094] When the video encoder receives the key frame generation message, determining a preset number of initial video frames obtained after receiving the key frame generation message as initial video frames corresponding to the preset driving scene;

[0095] Among them, the key frame generation message is triggered when the vehicle is in a preset driving scenario.

[0096] In some possible implementations, an acquisition module is further included, configured to:

[0097] After the image acquisition device in the vehicle is started, reading a first internal reference from the image acquisition device;

[0098] Control the image acquisition device to collect raw video data;

[0099] Based on the first internal reference, the original video data is processed to obtain a video stream.

[0100] In some possible implementations, a processing module is further included, configured to:

[0101] comparing the first internal reference with a pre-stored second internal reference;

[0102] When the first internal parameter is different from the pre-stored second internal parameter, the pre-stored second internal parameter is updated to the first internal parameter.

[0103] The functions and specific implementation principles of the above modules in the embodiments of the present disclosure can be referred to the above method embodiments and will not be repeated here.

[0104] The video stream processing device of the embodiment of the present disclosure adopts a video encoder to encode each initial video frame in the video stream to obtain the encoded video frame corresponding to each initial video frame, and then determines the target number of frames corresponding to each business type based on the sending frame rate corresponding to each business type and the initial frame rate corresponding to the video stream. Based on the target number of frames corresponding to each business type, the sending method of sending the encoded video frame to the video stream receiving module corresponding to each business type is determined, and finally, based on the sending method corresponding to each business type, the encoded video frame is sent to the video stream receiving module corresponding to each business type. Thus, after each video frame is encoded by the video encoder, the corresponding sending method is determined according to the video recording requirements of each business type, and then the encoded video frame is sent to the video stream receiving module corresponding to each business type, thereby not only reducing the resource usage of the video encoder, but also adapting to the video recording requirements of different business types.

[0105] Figure 4 4 is a functional block diagram of a vehicle, illustrating an exemplary embodiment. For example, vehicle 400 may be a hybrid vehicle, a non-hybrid vehicle, an electric vehicle, a fuel cell vehicle, or another type of vehicle. Vehicle 400 may be an autonomous vehicle, a semi-autonomous vehicle, or a non-autonomous vehicle.

[0106] Reference Figure 4 Vehicle 400 may include various subsystems, such as an infotainment system 410, a perception system 420, a decision-making control system 430, a drive system 440, and a computing platform 450. Vehicle 400 may also include more or fewer subsystems, and each subsystem may include multiple components. Furthermore, each subsystem and each component of vehicle 400 may be interconnected via wired or wireless means.

[0107] In some embodiments, the infotainment system 410 may include a communication system, an entertainment system, and a navigation system. The perception system 420 may include a number of sensors for sensing information about the environment surrounding the vehicle 400. For example, the perception system 420 may include a global positioning system (which may be a GPS system, a BeiDou system, or other positioning systems), an inertial measurement unit (IMU), a laser radar, a millimeter-wave radar, an ultrasonic radar, and a camera.

[0108] Decision control system 430 may include a computing system, a vehicle controller, a steering system, a throttle, and a braking system. Drive system 440 may include components that provide power and movement for vehicle 400. In one embodiment, drive system 440 may include an engine, an energy source, a transmission system, and wheels. The engine may be one or a combination of an internal combustion engine, an electric motor, or an air compression engine. The engine is capable of converting energy provided by the energy source into mechanical energy.

[0109] Some or all functions of the vehicle 400 are controlled by a computing platform 450. The computing platform 450 may include at least one processor 451 and a memory 452. The processor 451 may execute instructions 453 stored in the memory 452.

[0110] The processor 451 can be any conventional processor, such as a commercially available CPU. The processor can also include a graphics processor (GPU), a field programmable gate array (FPGA), a system on chip (SOC), an application specific integrated circuit (ASIC), or a combination thereof.

[0111] The memory 452 may be implemented by any type of volatile or non-volatile memory device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.

[0112] In addition to instructions 453, memory 452 may also store data such as road maps, route information, and vehicle location, direction, speed, and other data. The data stored in memory 452 may be used by computing platform 450. In the disclosed embodiment, processor 451 may execute instructions 453 to complete all or part of the steps of the aforementioned video stream processing method.

[0113] The present disclosure also provides a computer-readable storage medium having computer program instructions stored thereon. When the program instructions are executed by a processor, the steps of the video stream processing method provided by the present disclosure are implemented.

[0114] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present disclosure. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine different embodiments or examples described in this specification and features of different embodiments or examples, unless they are mutually inconsistent.

[0115] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features being referred to. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. Throughout the present disclosure, "plurality" means at least two, such as two, three, etc., unless otherwise specifically defined.

[0116] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code comprising one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present disclosure includes additional implementations in which functions may be performed out of the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present disclosure belong.

[0117] The logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing the logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (e.g., a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device). For purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include the following: an electrical connection with one or more wires (electronic devices), a portable computer disk cartridge (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and programmable read-only memory (EPROM or flash memory), fiber optic devices, and a portable compact disc read-only memory (CDROM). Furthermore, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing it in another suitable manner if necessary, and then storing it in a computer memory.

[0118] It should be understood that various parts of the present disclosure can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used to implement: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application-specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0119] Those skilled in the art will understand that all or part of the steps in the method of the above embodiment can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiment.

[0120] In addition, the functional units in the various embodiments of the present disclosure may be integrated into a single processing module, or each unit may exist physically separately, or two or more units may be integrated into a single module. The aforementioned integrated modules may be implemented in the form of hardware or in the form of software functional modules. If the integrated modules are implemented in the form of software functional modules and sold or used as independent products, they may also be stored in a computer-readable storage medium.

[0121] The storage medium mentioned above may be a read-only memory, a magnetic disk, or an optical disk, etc. Although the embodiments of the present disclosure have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present disclosure. A person of ordinary skill in the art may make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present disclosure.

Claims

1. A video stream processing method, characterized in that: include: Encoding each initial video frame in the video stream using a video encoder to obtain an encoded video frame corresponding to each initial video frame; Determining a target number of frames corresponding to each service type according to a sending frame rate corresponding to each service type and an initial frame rate corresponding to the video stream; Determining, based on a target number of frames corresponding to each of the service types, a sending mode for sending the encoded video frame to a video stream receiving module corresponding to each of the service types; Based on the sending mode corresponding to each of the service types, the encoded video frame is sent to the video stream receiving module corresponding to each of the service types.

2. The method according to claim 1, characterized in that The determining, based on the target number of frames corresponding to each of the service types, a sending method for sending the encoded video frame to the video stream receiving module corresponding to each of the service types includes: In response to the number of target frames corresponding to the first service type being multiple, determining the sending mode to compress the coded video frames of the target number of frames into one video segment and sending it to the video stream receiving module corresponding to the first service type; and / or, In response to the target number of frames corresponding to the second service type being one, the sending mode is determined to be sending each encoded video frame directly to the video stream receiving module corresponding to the second service type.

3. The method according to claim 1, characterized in that The step of encoding each initial video frame in the video stream using a video encoder to obtain an encoded video frame corresponding to each initial video frame includes: Encoding the initial video frame corresponding to the preset driving scene based on the encoding parameters corresponding to the key frame to obtain an encoded video frame; and / or, Based on a preset frame interval, a target video frame in the video stream is determined, and based on the encoding parameters, the target video frame is encoded to obtain an encoded video frame.

4. The method according to claim 3, characterized in that Also includes: When the video encoder receives a key frame generation message, determining a preset number of initial video frames acquired after receiving the key frame generation message as initial video frames corresponding to the preset driving scene; The key frame generation message is triggered when the vehicle is in the preset driving scenario.

5. The method according to claim 1, wherein Before encoding each initial video frame in the video stream using a video encoder to obtain an encoded video frame corresponding to each initial video frame, the method further includes: After the image acquisition device in the vehicle is started, reading a first internal reference from the image acquisition device; Controlling the image acquisition device to acquire original video data; The original video data is processed based on the first internal parameter to obtain the video stream.

6. The method according to claim 5, characterized in that After reading the first internal reference from the image acquisition device, the method further includes: comparing the first internal reference with a pre-stored second internal reference; When the first internal parameter is different from a pre-stored second internal parameter, the pre-stored second internal parameter is updated to the first internal parameter.

7. A video stream processing method and device, characterized in that: The device comprises: An encoding module, configured to encode each initial video frame in the video stream using a video encoder to obtain an encoded video frame corresponding to each initial video frame; A first determining module is configured to determine a target number of frames corresponding to each service type according to a sending frame rate corresponding to each service type and an initial frame rate corresponding to the video stream; A second determining module is configured to determine a sending mode for sending the encoded video frame to a video stream receiving module corresponding to each of the service types based on a target number of frames corresponding to each of the service types; The sending module is used to send the encoded video frame to the video stream receiving module corresponding to each of the service types based on the sending mode corresponding to each of the service types.

8. The device according to claim 7, characterized in that The second determining module is configured to: In response to the number of target frames corresponding to the first service type being multiple, determining the sending mode to compress the coded video frames of the target number of frames into one video segment and sending it to the video stream receiving module corresponding to the first service type; and / or, In response to the target number of frames corresponding to the second service type being one, the sending mode is determined to be sending each encoded video frame directly to the video stream receiving module corresponding to the second service type.

9. A vehicle, characterized in that: include: processor; A memory for storing processor-executable instructions; wherein the processor is configured to: implement the steps of the method according to any one of claims 1 to 6.

10. A non-transitory computer-readable storage medium, when instructions in the storage medium are executed by a processor of a mobile terminal, enables the mobile terminal to implement the steps of the method according to any one of claims 1 to 6.

Images