Vehicle-mounted video monitoring method and device

Abstract

The invention discloses a vehicle-mounted video monitoring method and device, and relates to the technical field of video image processing.The vehicle-mounted video monitoring method comprises the steps that a plurality of cameras are installed outside a vehicle, and a first video processing host responds to a trigger signal to obtain video data collected by each target camera in the same time period; and grouping the video data to obtain a first video group and a second video group, the installation positions of the cameras corresponding to the video data in each video group being adjacent in sequence. And the first video processing host and the second video processing host splice the first video group and the second video group respectively to obtain a first video processing result and a second video processing result. And splicing the first video processing result and the second video processing result to obtain panoramic video data, and transmitting the panoramic video data to a display device for display. According to the method, all video data are spliced into a panoramic video for a driver to check, so that the driver can accurately and timely master the panorama outside the vehicle, and the view blind area is eliminated.

Description

Technical Field

[0001] The present application relates to the technical field of video image processing, and more specifically, to a vehicle-mounted video monitoring method and device. Background Art

[0002] When a driver is driving a vehicle, the driver's line of sight and observation range are restricted or disturbed by factors such as the vehicle body structure, in-vehicle devices, light outside the vehicle, and road conditions, forming blind spots. To ensure driving safety, drivers usually use rearview mirrors, reversing radars, reversing images, etc. to expand the observation range outside the vehicle and reduce blind spots.

[0003] However, for multi-unit, long-body vehicles such as trams and articulated buses, due to the long body characteristics, even with the help of rearview mirrors, reversing radars and other equipment, the driver's observation range outside the vehicle is still limited and he cannot fully grasp the situation outside the vehicle. There is a large blind spot, and there are still great safety hazards in vehicle driving. Summary of the invention

[0004] In view of the above problems, the present application provides a vehicle-mounted video monitoring method and device to achieve the purpose of eliminating the driver's blind spot. The specific scheme is as follows:

[0005] In a first aspect, the present application provides a vehicle-mounted video monitoring method, which is applied to a first video processing host of a vehicle-mounted video monitoring system, wherein the vehicle-mounted video monitoring system further comprises: a second video processing host, a display device, and a plurality of cameras installed on the vehicle body, wherein the cameras are used to capture video data around the vehicle, and the plurality of cameras are respectively installed at the front of the vehicle and on both sides of the vehicle body according to a preset installation interval, and the shooting ranges of two cameras at adjacent installation positions partially overlap;

[0006] The vehicle-mounted video monitoring method comprises:

[0007] In response to a trigger signal, determining a target camera corresponding to the trigger signal, the trigger signal comprising at least: a power-on signal;

[0008] Acquire video data collected by each target camera in the same time period;

[0009] Grouping the video data to obtain a first video group and a second video group, wherein installation positions of cameras corresponding to the video data in each video group are adjacent in sequence;

[0010] Performing video stitching processing on the first video group to obtain a first video processing result;

[0011] Sending the second video group to the second video processing host for the video splicing processing, and obtaining a second video processing result sent by the second video processing host;

[0012] The video splicing process comprises: sequentially taking the video data in the video group as the data to be spliced ​​and splicing the corresponding preceding splicing data to obtain the video processing result corresponding to the video group, wherein the preceding splicing data is the splicing result of the video data before the video data in the video group;

[0013] Performing splicing processing on the first video processing result and the second video processing result to obtain panoramic video data;

[0014] The panoramic video data is transmitted to the display device for display.

[0015] In a possible implementation, sequentially taking the video data in the video group as the data to be spliced ​​and splicing them with the corresponding preceding splicing data to obtain the video processing result corresponding to the video group includes:

[0016] According to a preset image frame extraction rule, at least one frame of image is extracted from the data to be spliced ​​and the corresponding preceding spliced ​​data, and each frame of the image corresponds to a timestamp representing the shooting time;

[0017] Each frame of the image is tested respectively, and the tested image is used as the image to be spliced;

[0018] Identify a pattern formed by pixels in each frame of the image to be stitched;

[0019] For every two frames of the images to be spliced ​​with the same timestamp, similarity matching is performed on the graphs formed by the pixel points in the images to be spliced;

[0020] Splicing every two frames of the images to be spliced ​​that have the same timestamp and whose similarity of the graphics formed by the pixel points reaches a preset threshold, to obtain at least one frame of spliced ​​image;

[0021] Determining whether there is unjoined video data in the video group;

[0022] If the unstitched video data does not exist in the video group, generating a video stream based on all the stitched images to obtain a video processing result corresponding to the video group;

[0023] If the unspliced ​​video data exists in the video group, the spliced ​​image is used as the preceding spliced ​​data corresponding to the next data to be spliced.

[0024] In a possible implementation, the step of performing stitching processing on the first video processing result and the second video processing result to obtain panoramic video data includes:

[0025] Identify a preset master-slave relationship between the first video processing host and the second video processing host, wherein the backup video processing host in the master-slave relationship is used to perform hot standby redundancy for the main video processing host;

[0026] In the case where the first video processing host is the master video processing host relative to the second video processing host, the first video processing host respectively obtains stitched images in the first video processing result and the second video processing result, and a timestamp corresponding to each of the stitched images;

[0027] Identify a pattern formed by pixels in each of the stitched images;

[0028] Performing similarity matching between a graph formed by pixels in each of the stitched images and a graph formed by pixels in the stitched images with the same timestamp;

[0029] Stitching the two stitched images with the same timestamp and whose similarity of the graphics formed by the pixel points reaches a preset threshold, to obtain at least one frame of panoramic stitched image;

[0030] Based on all the panoramic stitched images, a video stream is generated to obtain panoramic video data.

[0031] In a possible implementation, the method further includes:

[0032] In the case where the first video processing host is a backup video processing host relative to the second video processing host, the first video processing host sends the first video processing result and the second video processing result to the second video processing host, so that the second video processing host executes the steps of respectively obtaining the stitched images in the first video processing result and the second video processing result, and the timestamp corresponding to each stitched image.

[0033] In a possible implementation, the method further includes:

[0034] If, among all the images to be stitched, there are no two frames of the images to be stitched that have the same timestamp and whose similarity of the patterns formed by the pixel points reaches a preset threshold, all the images to be stitched are deleted;

[0035] Extracting an image with the same timestamp from each of the video data in the video group as a target image;

[0036] Sorting the target images according to the arrangement order of the video data corresponding to the target images in the video group to obtain a target image sequence;

[0037] Sequentially taking the target images in the target image sequence as target images to be stitched, and performing stitching processing on the corresponding preceding stitched images, to obtain a partial field of view stitched image corresponding to the video group, wherein the preceding stitched image is a stitching result of the target images preceding the target image in the target image sequence;

[0038] Based on the partial field of view stitching image, a video processing result is generated.

[0039] In a possible implementation, the preset image frame extraction rule includes at least: the number of frames between each two extracted images.

[0040] In a possible implementation, the checking each frame of the image respectively and using the checked image as the image to be stitched includes:

[0041] Checking whether each frame of the image meets a preset condition, wherein the preset condition at least includes: the image does not have distortion or the brightness of the image reaches a preset brightness;

[0042] If the image meets the preset condition, the image meeting the preset condition is used as the image to be stitched;

[0043] If the image does not meet the preset condition, image parameters are adjusted for the image meeting the preset condition to obtain the image to be stitched.

[0044] In a possible implementation, when the trigger signal is a turn signal toward a target direction, the method further includes:

[0045] In response to the trigger signal, the camera that is biased toward the target direction is used as the target camera, and the steps of acquiring the video data collected by each of the target cameras in the same time period and subsequent steps are performed to obtain the panoramic video data of the vehicle in the target direction;

[0046] The panoramic video data is transmitted to the display device for display so that the driver can monitor the road conditions around the vehicle.

[0047] A second aspect of the present application provides a vehicle-mounted video monitoring device, which is applied to a first video processing host of a vehicle-mounted video monitoring system, wherein the vehicle-mounted video monitoring system further comprises: a second video processing host, a display device, and a plurality of cameras installed on the vehicle body, wherein the cameras are used to capture video data around the vehicle, and the plurality of cameras are respectively installed at the front of the vehicle and on both sides of the vehicle body according to a preset installation interval, and the shooting ranges of two cameras at adjacent installation positions partially overlap;

[0048] The vehicle-mounted video monitoring device comprises:

[0049] A signal trigger unit, configured to determine a target camera corresponding to the trigger signal in response to the trigger signal, wherein the trigger signal at least includes: a power-on signal;

[0050] A data acquisition unit, used to acquire video data collected by each target camera in the same time period;

[0051] A video grouping unit, used for grouping the video data to obtain a first video group and a second video group, wherein the installation positions of the cameras corresponding to the video data in each video group are adjacent in sequence;

[0052] A video processing unit, configured to perform video splicing processing on the first video group to obtain a first video processing result;

[0053] A video result acquisition unit, used for sending the second video group to the second video processing host for the video splicing process, and obtaining a second video processing result sent by the second video processing host;

[0054] The video splicing process comprises: sequentially taking the video data in the video group as the data to be spliced ​​and splicing the corresponding preceding splicing data to obtain the video processing result corresponding to the video group, wherein the preceding splicing data is the splicing result of the video data before the video data in the video group;

[0055] A video stitching unit, configured to stitch the first video processing result and the second video processing result to obtain panoramic video data;

[0056] A panoramic display unit is used to transmit the panoramic video data to the display device for display. As can be seen from the above technical solution, the embodiment of the present application installs multiple cameras outside the vehicle, and each camera captures video data from different angles outside the vehicle. Furthermore, the video data captured by the camera is grouped. Since the video data in each video group are captured by adjacent cameras, and the shooting ranges of adjacent cameras partially overlap, there are overlapping image features in each video data. The video data in each group is spliced ​​in sequence according to the order of adjacent videos to obtain the video processing results corresponding to the two groups, and then the two video processing results are spliced ​​to obtain panoramic video data. And the panoramic video data can be transmitted back to the display device for the driver to understand the situation outside the vehicle and eliminate blind spots in the field of vision.

[0057] The embodiment of the present application pre-joins all video data into a panoramic video for the driver to view. The driver does not need to manually combine multiple video images from different angles to determine the current driving conditions. The driver only needs to accurately and timely grasp the panoramic view outside the vehicle based on the panoramic video transmitted back, eliminating blind spots. In addition, the embodiment of the present application divides multiple video data into small groups and joins the videos in each small group in parallel. Compared with single-threaded joining of all video data, the speed of video joining can be increased, and the real-time monitoring of driving conditions can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0059] Figure 1 A schematic diagram of a camera installation provided in an embodiment of the present application;

[0060] Figure 2 A schematic diagram of a process flow of a vehicle-mounted video monitoring method provided in an embodiment of the present application;

[0061] Figure 3 An example diagram of video data splicing provided in an embodiment of the present application;

[0062] Figure 4 An example diagram of video data provided in an embodiment of the present application;

[0063] Figure 5 A schematic diagram of another process of vehicle-mounted video monitoring provided in an embodiment of the present application;

[0064] Figure 6A schematic diagram of the structure of a vehicle-mounted video surveillance device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0065] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0066] For multi-unit, long-body vehicles, when the vehicle is driving, especially when turning, the driver cannot fully grasp the conditions outside the vehicle through the rearview mirror on the turning side and the rear side of the vehicle due to the tilt of the body, resulting in blind spots in the driver's vision of the current road conditions. Accidents such as scratches and collisions may occur during driving, affecting the safety of people in the vehicle.

[0067] In order to eliminate the blind spots of the driver's field of vision, an embodiment of the present application proposes a vehicle-mounted video monitoring system, including: multiple cameras installed on the vehicle body, the cameras are used to capture video data around the vehicle, the multiple cameras are installed on the front of the vehicle and both sides of the vehicle body according to preset installation intervals, and the shooting ranges of two cameras with adjacent installation positions partially overlap; a first video processing host and a second video processing host, which are used to implement a vehicle-mounted video monitoring method, process the video data collected by the cameras, and obtain a panoramic video; a display device, which is used to display the panoramic video sent by the video processing module.

[0068] Reference Figure 1 , a schematic diagram of camera installation provided in an embodiment of the present application, this embodiment takes a tram with 5 cars as an example to illustrate a vehicle-mounted video surveillance system for realizing panoramic surround monitoring. Among them, multiple cameras include: front cameras BSXJ1 and FSXJ1, right side cameras RSXJ1, RSXJ2, RSXJ3, RSXJ4, and left side cameras LSXJ1, LSXJ2, LSXJ3, LSXJ4. It can be understood that the camera is arranged in a position where it can capture the surroundings of the vehicle body, so the camera's field of view can completely cover the surroundings of the vehicle body, and based on the video data collected by these cameras, a panoramic video or panoramic image can be obtained.

[0069] Among them, each camera installed on the left side of the car is installed according to the preset installation interval, but in order to facilitate the subsequent processing of video data, the shooting ranges of each two adjacent cameras must overlap. Specifically, the camera model, lens parameters, quantity, preset installation interval, etc. can be determined according to the length of the vehicle body, budget, etc. For example, the cameras in this example all use a wide-angle camera with a horizontal viewing angle of 190°, a diagonal viewing angle of 200°, and a viewing distance of 10 meters. The length of the vehicle is 33 meters. According to the viewing distance of the camera and the overlapping shooting ranges of two adjacent cameras, it is determined that the distance between adjacent cameras can be 8 meters. It can be determined that 4 cameras can be installed on each side of the vehicle body, and a total of 10 cameras are required.

[0070] The first video processing host, the second video processing host and the display device can be installed in the driver's cab. The display device can be the existing central control display screen in the car, or a separately set up LCD display screen, which is used to display panoramic video data for the driver to view and eliminate blind spots in the field of vision.

[0071] Reference Figure 2 , a flow chart of a vehicle-mounted video monitoring method provided in an embodiment of the present application illustrates the vehicle-mounted video monitoring method applied to a first video processing host, wherein the specific process may include the following contents.

[0072] Step S110, in response to the trigger signal, determining a target camera corresponding to the trigger signal.

[0073] The trigger signal at least includes: a power-on signal. Optionally, the trigger signal may also include: a left or right turn signal, a reverse signal, etc.

[0074] The vehicle's power-on signal proves that the vehicle is currently in driving state, and it is necessary to obtain the road conditions around the vehicle in real time. In the embodiment of the present application, the power-on signal can be defaulted to a straight-ahead signal. When the driver is driving straight, he needs to observe all conditions in front of, behind, and on both sides of the vehicle. Therefore, the target camera corresponding to the power-on signal is all cameras installed on the vehicle body.

[0075] If the trigger signal is a turn signal to the left or right, for safety reasons, the driver is more concerned about the external real-time situation on the turning side. Therefore, the embodiment of the present application determines the turning direction as the target direction, responds to the trigger signal of the turn signal toward the target direction, uses the camera that is biased toward the target direction as the target camera, and executes subsequent steps S120-S170. Figure 1For example, if the driver triggers the right turn signal, the vehicle computer or signal light will simultaneously send the right turn signal to the video processing module, and the video processing module will identify the right-side cameras RSXJ1, RSXJ2, RSXJ3, RSXJ4, and BSXJ1 as the target cameras. It is understandable that due to the existence of the turning radius, when turning, a multi-group, long-body vehicle will have a certain angle between the carriages, and the overlapping area of ​​the camera shooting area will become larger. In the subsequent video data splicing and fusion process, it can provide the driver with a clearer video of the turning side, which is convenient for the driver to accurately grasp the situation outside the vehicle.

[0076] Similarly, if the trigger signal is a reverse signal, the camera that can capture the rear of the vehicle, such as LSXJ4, RSXJ4 and BSXJ1, is determined as the target camera. In the embodiment of the present application, the target camera corresponding to different trigger signals is not limited, and can be determined according to the driver's needs for the camera's capture range.

[0077] Step S120, obtaining video data collected by each target camera in the same time period.

[0078] The embodiment of the present application takes the trigger signal as a power-on signal as an example to obtain the video data collected by all cameras in the vehicle within the same time period. It can be understood that since the panoramic view outside the vehicle is transmitted back to the driver in real time, it is necessary to frequently obtain video data in a short period of time and process the video data quickly so that it can be transmitted back in time to avoid delays between the transmitted panoramic view and the current real scene outside the vehicle.

[0079] In this example, the video data collected by each target camera within one second is obtained, and the video data collected within one second is subsequently processed to obtain a panoramic video.

[0080] Step S130, grouping the video data to obtain a first video group and a second video group, wherein the installation positions of the cameras corresponding to the video data in each video group are adjacent in sequence.

[0081] Considering the large amount of video data, if they are stitched and fused together, the calculations will be complex and the amount of calculation will be large, which will cause delays. Therefore, the embodiment of the present application divides the video data into multiple video groups, each video group contains a small amount of video data, and uses each video group as a processing unit to perform video stitching in a multi-threaded manner, which can greatly improve the efficiency of video stitching.

[0082] The embodiment of the present application takes into account that most multi-group, long-body vehicles have a vehicle system installed at the front and rear of the vehicle, respectively, as the first video processing host and the second video processing host. Therefore, the embodiment of the present application divides the video data into two video groups, which are processed by the first video processing host and the second video processing host respectively. If the vehicle has multiple functional modules or devices that can be used for video processing, the video data can also be divided into more video groups for processing.

[0083] When installing the cameras, the shooting ranges of adjacent cameras are partially overlapped. This is to splice the overlapping videos when splicing the videos, so as to reduce the misalignment of the spliced ​​videos. When the video is subsequently spliced ​​using video groups as units, the video data in the video group should also be collected by adjacent cameras. Therefore, the video data captured by the cameras are grouped according to the positional relationship between the cameras. Figure 1 For example, the video data shot by FSXJ1, LSXJ1, LSXJ2, LSXJ3, and LSXJ4 are divided into a group, and the installation positions of the five cameras FSXJ1, LSXJ1, LSXJ2, LSXJ3, and LSXJ4 are adjacent in sequence, that is, FSXJ1 is adjacent to LSXJ1, LSXJ1 is adjacent to LSXJ2, LSXJ2 is adjacent to LSXJ3, and LSXJ3 is adjacent to LSXJ4.

[0084] It can be seen that the original video processing module needs to splice 10 video data at a time. After grouping, it only needs to splice 5 video data in parallel, splice the 5 video data into 1 video, and reconstruct and splice the 1 video spliced ​​with the other 5 video data, and synthesize and process it into a complete panoramic video. Compared with directly splicing 10 video data, the calculation complexity is reduced, and multi-threaded video splicing improves the efficiency of video splicing.

[0085] When the video data is subsequently stitched together, it is necessary to ensure that there are overlapping or similar scenes in the image pictures reflected by any two video data being stitched together, and use this as the basis for video stitching to avoid video stitching misalignment. Therefore, it is necessary to ensure that the two videos being stitched together each time must be shot by adjacent cameras. Therefore, the installation positions of the cameras corresponding to any two adjacent video data in the video data sequence within the video group are adjacent, thereby improving the accuracy of video stitching.

[0086] It is understandable that the embodiment of the present application takes into account that most multi-group, long-body vehicles have a vehicle system installed at the front and rear of the vehicle, respectively, as the first video processing host and the second video processing host. Therefore, the embodiment of the present application divides the video data into two video groups, which are processed by the first video processing host and the second video processing host respectively. If the vehicle has multiple functional modules or devices that can be used for video processing, the video data can also be divided into more video groups for processing. The embodiment of the present application does not limit the number of video groups, which can be determined adaptively according to the number of video processing hosts, the number of parallel threads allowed by the video processing hosts, the number of videos, and the user's requirements for video processing efficiency, speed, etc.

[0087] Step S140: performing video splicing processing on the first video group to obtain a first video processing result.

[0088] Step S150, sending the second video group to the second video processing host for video splicing processing, and obtaining a second video processing result sent by the second video processing host.

[0089] The video splicing process includes: taking the video data in the video group as the data to be spliced ​​in sequence, and splicing it with the corresponding preceding splicing data to obtain the video processing result corresponding to the video group, wherein the preceding splicing data is the splicing result of the video data before the video data in the video group.

[0090] In the embodiment of the present application, the video data in the video group are spliced ​​in sequence. Figure 3 , an example diagram of video data stitching provided in an embodiment of the present application illustrates the process of video stitching processing in steps S140 and S150.

[0091] In the figure, A1, B1, C1, and D1 are video data captured by the left side cameras LSXJ1, LSXJ2, LSXJ3, and LSXJ4 of the vehicle respectively. Since the left side cameras LSXJ1, LSXJ2, LSXJ3, and LSXJ4 of the vehicle are adjacent cameras in sequence, the video data sequence corresponding to the video group is: A1, B1, C1, and D1.

[0092] As can be seen from the figure, there are overlapping or similar scenes in the video data captured by any two adjacent cameras, such as the content framed by dotted lines in A1 and B1 is the overlapping content. Therefore, based on the overlapping content of A1 and B1, the video data A1 and B1 can be spliced ​​to obtain the scene captured by cameras LSXJ1 and LSXJ2, that is, video data AB.

[0093] In order to improve the efficiency of video splicing within a video group and avoid the problem of image misalignment caused by disordered splicing, the embodiment of the present application performs splicing in sequence according to the order in which the videos are arranged in the video group. When it is the turn of the video data to be spliced, the splicing process should be performed on the spliced ​​previous splicing data to reduce the situation of repeated splicing of video data. For example, when it is the turn of the video C1 shot by LSXJ3 as the data to be spliced, C1 is spliced ​​with the previous splicing data AB to obtain the scene outside the car shot by cameras LSXJ1, LSXJ2, and LSXJ3, that is, video data ABC. Otherwise, if B1 and C1 are spliced, C1 and D1 are spliced, and then the three spliced ​​video data are spliced, the amount of video splicing calculations will increase, and repeated splicing will lead to too many useless calculations, reducing the splicing efficiency of video data.

[0094] Based on this, further, the video data ABC is spliced ​​with the video data D1 to obtain ABCD, and then spliced ​​with the camera BSXJ1 to obtain ABCDE, and the panoramic view of the left side and rear side of the vehicle body taken by the left side cameras LSXJ1, LSXJ2, LSXJ3, LSXJ4 and the rear side BSXJ1 is obtained, that is, the scene shown by the video data ABCDE, and the video data ABCDE is used as the left side field of view video data of the vehicle, that is, the first video processing result.

[0095] It should be noted that the preceding splicing data is the splicing result of the video data before the video data in the video group, and for the first video data in the video data sequence in the video group, the preceding splicing data is blank video data, and for the second video data in the video data sequence, the preceding splicing data is the splicing result of the blank video data and the first video data, that is, the first video data.

[0096] It can be understood that the video data captured by other cameras are also subjected to the above-mentioned stitching processing in other video groups at the same time, that is, the second video processing host performs the same processing on the second video group captured by the right side cameras RSXJ1, RSXJ2, RSXJ3, and RSXJ4 of the vehicle to obtain the corresponding right side field of view video data of the vehicle, that is, the second video data processing result.

[0097] Step S160: stitching the first video processing result and the second video processing result to obtain panoramic video data.

[0098] Step S170: transmitting the panoramic video data to a display device for display.

[0099] By stitching together the video data of each side of the vehicle, panoramic video data of the vehicle body can be obtained, and the obtained panoramic video data can be sent to a display device for display, so that the driver can grasp the scene outside the vehicle in time, broaden the driver's observation range, and improve driving safety.

[0100] Therefore, the second video processing host needs to send the second video processing result to the first video processing host, so that the first video processing host splices the first video processing result with the second video processing result to obtain panoramic video data. Optionally, the second video processing host can also receive the first video processing result sent by the first video processing host, and the second video processing host splices the first video processing result with the second video processing result.

[0101] In summary, the embodiment of the present application installs multiple cameras outside the vehicle, and each camera captures video data from different angles outside the vehicle. Furthermore, the video data captured by the camera is grouped. Since the video data in each video group is captured by adjacent cameras, and the shooting ranges of adjacent cameras partially overlap, there are overlapping image features in each video data. The video data in each group is spliced ​​in sequence according to the order of adjacent videos to obtain the video processing results corresponding to the two groups, and then the two video processing results are spliced ​​to obtain panoramic video data. In addition, the panoramic video data can be transmitted back to the display device for the driver to understand the situation outside the vehicle and eliminate blind spots in the field of vision.

[0102] The embodiment of the present application pre-joins all video data into a panoramic video for the driver to view. The driver does not need to manually combine multiple video images from different angles to determine the current driving conditions. The driver only needs to accurately and timely grasp the panoramic view outside the vehicle based on the panoramic video transmitted back, eliminating blind spots. In addition, the embodiment of the present application divides multiple video data into small groups and joins the videos in each small group in parallel. Compared with single-threaded joining of all video data, the speed of video joining can be increased, and the real-time monitoring of driving conditions can be improved.

[0103] Next, other possible implementations of the vehicle-mounted video surveillance method provided in the embodiments of the present application are described.

[0104] In a possible implementation, the process of performing splicing processing on any to-be-spliced ​​video and the previous splicing data in step S140 may include:

[0105] According to the preset image frame extraction rule, at least one frame of image is extracted from the data to be spliced ​​and the corresponding preceding spliced ​​data, and each frame of image corresponds to a timestamp representing the shooting time; each frame of image is inspected respectively, and the inspected image is used as the image to be spliced; the pattern formed by the pixel points in each frame of image to be spliced ​​is identified; for every two frames of image to be spliced ​​with the same timestamp, the pattern formed by the pixel points in the image to be spliced ​​is matched for similarity; every two frames of image to be spliced ​​with the same timestamp and the similarity of the pattern formed by the pixel points reaches a preset threshold value are spliced ​​to obtain at least one frame of spliced ​​image; and it is determined whether there is unspliced ​​video data in the video group.

[0106] If there is no unspliced ​​video data in the video group, a video stream is generated based on all the spliced ​​images to obtain a video processing result corresponding to the video group; if there is unspliced ​​video data in the video group, the spliced ​​image is used as the preceding spliced ​​data corresponding to the next data to be spliced.

[0107] Video data is a dynamic picture formed by a series of continuous images played quickly in a specific order and time interval. Figure 4 , an example diagram of video data provided in an embodiment of the present application, video data A, B, C, and D contain multiple frames of images, and due to the shooting frequency of the camera installed on the vehicle body in the embodiment of the present application, and the video data collected are of the same time period, the number of frames of images contained in the video data A, B, C, and D is the same, and the time interval between each frame of the video data is the same.

[0108] The essence of video is image, and the pictures taken by the camera in a short period of time are the same. Therefore, the embodiment of the present application extracts a small number of images for stitching, and can obtain the same scene displayed by the video data obtained by stitching each frame of the image in the video data. Based on this, the efficiency of video stitching can be improved.

[0109] Optionally, images are extracted from the data to be spliced ​​and the previous spliced ​​data respectively according to the same preset image frame extraction rule, wherein the preset image frame extraction rule at least includes: the number of frames between each two extracted images, for example: for video data with 30 frames per second, 1 frame of image is extracted every 5 frames, therefore, 6 frames of image can be extracted from 1 second of the data to be spliced.

[0110] Furthermore, during the camera shooting process, the captured image may be distorted or out of focus due to the shaking or driving of the vehicle body. Therefore, each frame of the image needs to be checked to ensure that the stitched panoramic video can clearly reflect the scene outside the vehicle.

[0111] In a possible implementation, each frame of the image is checked respectively, and the checked image is used as the image to be stitched, including: checking whether each frame of the image meets preset conditions, and the preset conditions at least include: there is no distortion in the image or the brightness of the image reaches a preset brightness; if the image meets the preset conditions, the image meeting the preset conditions is used as the image to be stitched; if the image does not meet the preset conditions, the image parameters of the image meeting the preset conditions are adjusted to obtain the image to be stitched.

[0112] In the embodiment of the present application, the preset conditions for inspecting the image may include: image focus, image brightness, and image distortion degree meeting the requirements. If the image fails to meet the requirements, the image can be corrected through custom parameters, and the corrected image can be used as the image to be stitched for subsequent stitching to improve the accuracy of the panoramic video.

[0113] In this example, video data B is the data to be spliced, and video data A is the previous splicing result, then at least one frame of image is extracted therefrom respectively, for example, the data to be spliced ​​extracted from video data A are: A1, A2, A3, A4, and the images to be spliced ​​extracted from video data B are B1, B2, B3, B4, and since video data A and video data B are video data in the same time period and have the same image extraction rules, the timestamps corresponding to A1 and B1 are the same, and similarly, the timestamps corresponding to A2 and B2, A3 and B3, A4 and B4 are also the same.

[0114] Reference Figure 4 , there are overlapping or similar scenes in any two adjacent images of video data, and the embodiment of the present application splices the images according to the similar scenes. Therefore, the pixels in each frame of the image to be spliced ​​are identified, and the figure formed by the pixels is determined. By matching the overlap or similarity of the figures formed by the pixels in the two images, it is determined whether there is scene overlap, and the point of image splicing is determined. Considering that only the scenes captured by the camera at the same time point belong to the scenes at the same moment, there will be a common field of view. Therefore, the embodiment of the present application requires that for every two frames of images to be spliced ​​with the same timestamp, the figures formed by the pixels in the images to be spliced ​​are matched for similarity, that is, A1 can only be matched with B1.

[0115] Reference Figure 3In the figure, A1 and B1 are the images to be spliced ​​extracted from video data A and video data B respectively. Among them, the graphics formed by the pixels in the dotted box are almost the same, and the similarity exceeds the preset threshold. A1 and B1 can be spliced ​​to obtain AB1; at the same time, A2 and B2, A3 and B3, A4 and B4 are also matched for similarity, and spliced ​​to obtain spliced ​​images. Assuming that the similarity of the graphics formed by the pixels corresponding to A2 and B2, A3 and B3, A4 and B4 all reach the preset threshold, they are spliced ​​to obtain spliced ​​images AB2, AB3, and AB4.

[0116] Based on this, video data A and video data B complete video splicing, and determine whether there is any unspliced ​​video data in the video group. If not, AB1, AB2, AB3, and AB4 are generated into a video stream according to the time sequence of the corresponding timestamps to obtain video data showing the left side field of view of the vehicle body, that is, the video processing result corresponding to the video group containing video data A and video data B.

[0117] If there is still unjoined video data in the video group, AB1, AB2, AB3, and AB4 are used as the preceding joint data, and the joint processing of the video data C is continued. It can be understood that, since AB1, AB2, AB3, and AB4 have corresponding timestamps, when extracting images from the video data C, images with the same timestamps as AB1, AB2, AB3, and AB4 can be directly extracted to avoid extracting too many images, which will make it impossible to perform image matching and increase unnecessary calculations.

[0118] In another possible implementation, if among all the images to be stitched, there are no two frames of images to be stitched with the same timestamp and the similarity of the graphics formed by the pixel points reaches a preset threshold, all the images to be stitched are deleted; from each video data in the video group, an image with the same timestamp is extracted as the target image; according to the arrangement order of the video data corresponding to the target image in the video group, the target images are sorted to obtain a target image sequence; the target images in the target image sequence are sequentially used as target images to be stitched, and stitched with their corresponding preceding stitched images to obtain a partial field of view stitched image corresponding to the video group, where the preceding stitched image is the stitching result of the target images before the target image in the target image sequence; based on the partial field of view stitched image, a video processing result is generated.

[0119] If each frame of the image to be spliced ​​of each video data in the video group does not satisfy the above matching, all the extracted images to be spliced ​​will be deleted, and any frame of the first video data in the video group will be extracted. Taking the above video group as an example, video data A is extracted to obtain A'. Further, images B', C', and D' with the same timestamp as A' are extracted from video data B, C, and D, respectively, as target images.

[0120] Further, according to the arrangement order of the video data A, B, C, D in the video group, A', B', C', D' are spliced, referring to Figure 2 , I will not repeat it here, except that this stitching does not consider the similarity of the graphics formed by the pixels of the image, and the stitching result obtained is a partial field of view stitching image. The partial field of view stitching image is converted into a video stream to obtain a video processing result.

[0121] In a possible implementation, at least two video processing results are stitched together to obtain panoramic video data. In this embodiment, the first video processing result and the second video processing result are stitched together to obtain the panoramic video data, including: identifying a preset master-slave relationship between a first video processing host and a second video processing host, wherein the backup video processing host in the master-slave relationship is used to perform hot standby redundancy for the master video processing host; when the first video processing host is the master video processing host relative to the second video processing host, the first video processing host respectively obtains stitched images in the first video processing result and the second video processing result, and a timestamp corresponding to each stitched image; identifying a figure formed by pixels in each stitched image; performing similarity matching between a figure formed by pixels in each stitched image and a figure formed by pixels in a stitched image with the same timestamp; stitching two stitched images with the same timestamp and whose similarity of the figures formed by pixels reaches a preset threshold to obtain at least one frame of panoramic stitched image; generating a video stream based on all the panoramic stitched images to obtain panoramic video data.

[0122] When the first video processing host is a backup video processing host relative to the second video processing host, the first video processing host sends the first video processing result and the second video processing result to the second video processing host, so that the second video processing host executes the steps of respectively obtaining the stitched images in the first video processing result and the second video processing result, as well as the timestamp corresponding to each stitched image and subsequent steps.

[0123] The embodiment of the present application pre-sets the master-slave relationship between the first video processing host and the second video processing host, and the master and backup video processing hosts are hot-standby redundant, that is, when the master video processing host is working normally, the backup video processing host is always in a ready state, and synchronizes data with the master video processing host through a dedicated channel. Once the master video processing host fails, the backup video processing host will immediately take over the video splicing work of the master video processing host, ensuring the continuous operation of the vehicle-mounted video monitoring method, and the user will not perceive the downtime caused by the failure of the master video processing host.

[0124] In order not to affect hot standby redundancy, the first video processing host identifies whether it is the main video processing host before splicing the video processing results, so as to avoid using the backup video processing host to work while the main video processing host cannot synchronize data, resulting in the inability to switch the processing host when a fault occurs.

[0125] If the first video processing host is not the main video processing host, the first video processing host synchronizes all the video processing results of the host to the second video processing host, and the second video processing host performs splicing processing.

[0126] In the embodiment of the present application, taking the case where the first video processing host is the main video processing host as an example, the splicing process of the first video processing result and the second video processing result is described as follows:

[0127] It can be understood that the video processing result is generated by at least one stitched image, so each frame image extracted from the video processing result is the stitched image obtained by the above stitching. Referring to the above stitching method for the images in the video data to be stitched and the previous stitching data, it can also be applied to the stitching process of the stitched images in the video processing result.

[0128] Since the video processing result is the splicing result obtained from each video group, it is relatively small compared to the amount of video data, and there will not be too much time difference if they are not sorted and matched in sequence. Therefore, the image formed by the pixels in each spliced ​​image can be directly identified, and the spliced ​​images of the spliced ​​data belonging to the same timestamp can be matched for similarity of the images formed by the pixels, to determine whether each frame of the spliced ​​image in different video processing results can be spliced, and if it can be spliced, determine the splicing position points according to the pixels.

[0129] Furthermore, two stitched images with the same timestamp and whose pixel-point-formed graphics similarity reaches a preset threshold value in all video processing results are stitched together to obtain at least one panoramic stitched image. All panoramic stitched images are converted into video streams to obtain panoramic video data.

[0130] Optionally, during the stitching process, the stitching result may also be processed to eliminate stitching marks and image stitching seams, so as to improve the accuracy and aesthetics of the panoramic video displayed to the driver.

[0131] Reference Figure 5 , a flow chart of another vehicle-mounted video monitoring method provided in an embodiment of the present application is used to exemplify the above-mentioned vehicle-mounted video monitoring method.

[0132] Among them, the video processing host has the ability to process multiple video streams, analyze video distortion, adjust brightness, extract features, image fusion and other functions, as well as the ability to encode and store. Based on this, the video processing host can fuse and splice multiple videos into panoramic images or panoramic videos. Optionally, in this example, both the first video processing host and the second video processing host can be equipped with an ARM Cortex-A9 processor, support 6×AHD video signal access, and support 2×AHD video signal output. The video acquisition unit is 10 wide-angle cameras with a horizontal viewing angle of 195°, a vertical viewing angle of 140°, a resolution of 1280×720, and a viewing distance of 10 meters. The display module uses an LCD display with a resolution of 1280×720. Based on the 1080P video stream with the largest bandwidth occupancy, the bandwidth required for a single camera does not exceed 5M / s, so 100M Ethernet is used as the video transmission medium.

[0133] In this example, two video processing hosts are installed in the vehicle, and each video processing host is responsible for the splicing processing of 1 / 2 of the video data of the whole vehicle. As shown in the figure, the first video splicing unit of the first video processing host is responsible for the splicing processing of the video data collected by the FSXJ1, LSXJ1, LSXJ2, LSXJ3, and LSXJ4 cameras, and the second video splicing unit of the second video processing host is responsible for the splicing processing of the video data collected by the BSXJ1, RSXJ1, RSXJ2, RSXJ3, and RSXJ4 cameras.

[0134] Furthermore, the first video processing host and the second video processing host perform image processing on the acquired 5-channel video data respectively, implement the above-mentioned vehicle-mounted video monitoring method, and obtain the first video processing result A and the second video processing result B respectively. Then, the second video processing host performs image processing on the two video processing results according to image feature point extraction, matching, and timestamp, realizes secondary reconstruction and splicing, synthesizes and processes the two video processing results into a complete panoramic video stream, and finally outputs it to the display module where the driver is located for display, and stores it in the main video processing host or the storage submodule of the slave video processing host for the driver to view the driving record.

[0135] The vehicle-mounted video monitoring device provided in an embodiment of the present application is described below. The vehicle-mounted video monitoring device described below and the vehicle-mounted video monitoring method described above can be referenced to each other.

[0136] First, combine Figure 6 , the vehicle-mounted video monitoring device of the video processing module applied to the vehicle-mounted video monitoring system is introduced, such as Figure 6 As shown, the vehicle-mounted video monitoring device may include:

[0137] The signal trigger unit 100 is used to determine a target camera corresponding to the trigger signal in response to the trigger signal, wherein the trigger signal at least includes: a power-on signal;

[0138] A data acquisition unit 200 is used to acquire video data collected by each target camera in the same time period;

[0139] The video grouping unit 300 is used to group the video data to obtain a first video group and a second video group, wherein the installation positions of the cameras corresponding to the video data in each video group are adjacent in sequence;

[0140] The video processing unit 400 is used to perform video splicing processing on the first video group to obtain a first video processing result;

[0141] The video result acquisition unit 500 is used to send the second video group to the second video processing host for the video splicing process, and obtain the second video processing result sent by the second video processing host;

[0142] The video splicing process comprises: sequentially taking the video data in the video group as the data to be spliced ​​and splicing the corresponding preceding splicing data to obtain the video processing result corresponding to the video group, wherein the preceding splicing data is the splicing result of the video data before the video data in the video group;

[0143] A video stitching unit 600 is used to stitch the first video processing result and the second video processing result to obtain panoramic video data;

[0144] The panoramic display unit 700 is used to transmit the panoramic video data to the display device for display.

[0145] In summary, the embodiment of the present application installs multiple cameras outside the vehicle, and each camera captures video data from different angles outside the vehicle. Furthermore, the video data captured by the camera is grouped. Since the video data in each video group is captured by adjacent cameras, and the shooting ranges of adjacent cameras partially overlap, there are overlapping image features in each video data. The video data in each group is spliced ​​in sequence according to the order of adjacent videos to obtain the video processing results corresponding to the two groups, and then the two video processing results are spliced ​​to obtain panoramic video data. In addition, the panoramic video data can be transmitted back to the display device for the driver to understand the situation outside the vehicle and eliminate blind spots in the field of vision.

[0146] The embodiment of the present application pre-joins all video data into a panoramic video for the driver to view. The driver does not need to manually combine multiple video images from different angles to determine the current driving conditions. The driver only needs to accurately and timely grasp the panoramic view outside the vehicle based on the panoramic video transmitted back, eliminating blind spots. In addition, the embodiment of the present application divides multiple video data into small groups and joins the videos in each small group in parallel. Compared with single-threaded joining of all video data, the speed of video joining can be increased, and the real-time monitoring of driving conditions can be improved.

[0147] In a possible implementation, the video processing unit 400 includes:

[0148] An image extraction subunit, configured to extract at least one frame of image from the data to be spliced ​​and the corresponding preceding spliced ​​data respectively according to a preset image frame extraction rule, and each frame of the image corresponds to a timestamp representing a shooting time;

[0149] An image inspection subunit, used to inspect each frame of the image respectively, and use the inspected image as the image to be spliced;

[0150] A graphic recognition subunit, used for recognizing a graphic formed by pixels in each frame of the image to be spliced;

[0151] A matching subunit, configured to perform similarity matching on the graphs formed by the pixel points in the images to be spliced ​​for every two frames of the images to be spliced ​​with the same timestamp;

[0152] A stitching subunit, configured to stitch every two frames of the images to be stitched that have the same timestamp and whose similarity of the graphics formed by the pixel points reaches a preset threshold, to obtain at least one frame of stitched image;

[0153] A judging subunit, used for judging whether there is any unjoined video data in the video group;

[0154] A video generation subunit, configured to generate a video stream based on all the stitched images to obtain a video processing result corresponding to the video group when the judgment result of the judgment subunit is negative;

[0155] The continuing stitching subunit is used for, when the judgment result of the judging subunit is yes, using the stitched image as the preceding stitched data corresponding to the next data to be stitched.

[0156] In a possible implementation, the video splicing unit 600 includes:

[0157] A master-slave relationship identification subunit, used to identify a preset master-slave relationship between the first video processing host and the second video processing host, wherein the backup video processing host in the master-slave relationship is used to perform hot standby redundancy for the master video processing host;

[0158] a stitched image acquisition subunit, configured to, when the recognition result of the master-slave relationship recognition subunit is that the first video processing host is the master video processing host relative to the second video processing host, cause the first video processing host to respectively acquire the stitched images in the first video processing result and the second video processing result, and a timestamp corresponding to each stitched image;

[0159] A pixel point recognition subunit, used for recognizing a pattern formed by pixels in each of the stitched images;

[0160] a similarity matching subunit, configured to perform similarity matching between a graph formed by pixels in each of the stitched images and a graph formed by pixels in the stitched images having the same timestamp;

[0161] A panoramic image stitching subunit, configured to stitch the two stitched images having the same timestamp and whose similarity of the graphics formed by the pixel points reaches a preset threshold, to obtain at least one frame of panoramic stitching image;

[0162] The panoramic video generation subunit is used to generate a video stream based on all the panoramic stitched images to obtain panoramic video data.

[0163] In a possible implementation, the video splicing unit 600 further includes:

[0164] A data sending subunit is used for sending the first video processing result and the second video processing result to the second video processing host when the identification result of the master-slave relationship identification subunit is that the first video processing host is a backup video processing host relative to the second video processing host, so that the second video processing host executes the steps of respectively obtaining the stitched images in the first video processing result and the second video processing result, and the timestamp corresponding to each stitched image.

[0165] In a possible implementation, the video processing unit 400 further includes:

[0166] An image deletion subunit, configured to delete all the images to be spliced ​​if, among all the images to be spliced, there are no two frames of the images to be spliced ​​that have the same timestamp and whose similarity of the patterns formed by the pixel points reaches a preset threshold;

[0167] A target image extraction subunit, used to extract an image with the same timestamp from each of the video data in the video group as a target image;

[0168] An image sorting subunit, used for sorting the target images according to the arrangement order of the video data corresponding to the target images in the video group to obtain a target image sequence;

[0169] The image sequential stitching subunit is used to sequentially use the target image in the target image sequence as the target image to be stitched, and stitch it with the corresponding preceding stitching image to obtain a partial field of view stitching image corresponding to the video group, wherein the preceding stitching image is the stitching result of the target images before the target image in the target image sequence;

[0170] The video generation subunit is used to generate a video processing result based on the partial field of view stitching image.

[0171] In a possible implementation, the preset image frame extraction rule includes at least: the number of frames between each two extracted images.

[0172] In a possible implementation, the image inspection subunit includes:

[0173] A checking subunit, used to check whether each frame of the image meets a preset condition, wherein the preset condition at least includes: the image has no distortion or the brightness of the image reaches a preset brightness;

[0174] An image adjustment subunit, configured to adjust image parameters of the image satisfying the preset condition to obtain an image to be spliced ​​if the inspection result of the inspection subunit is negative;

[0175] The image confirmation subunit is used to use the image that meets the preset condition as the image to be spliced ​​when the inspection result of the inspection subunit is yes.

[0176] In a possible implementation, when the trigger signal is a turn signal toward a target direction, the method further includes:

[0177] A turn signal trigger unit, configured to respond to the trigger signal, use the camera that is biased toward the target direction as a target camera, and jump to the data acquisition unit 200 to execute the steps of acquiring the video data collected by each of the target cameras in the same time period and the subsequent steps, so as to obtain the panoramic video data of the vehicle in the target direction;

[0178] The video display unit is used to transmit the panoramic video data to the display device for display so that the driver can monitor the road conditions around the vehicle.

[0179] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0180] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0181] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A vehicle-mounted video monitoring method, characterized in that: A first video processing host applied to a vehicle-mounted video monitoring system, the vehicle-mounted video monitoring system further comprising: a second video processing host, a display device, and a plurality of cameras installed on the vehicle body, the cameras being used to capture video data around the vehicle, the plurality of cameras being installed at the front of the vehicle and both sides of the vehicle body according to a preset installation interval, and the shooting ranges of two cameras installed at adjacent positions partially overlap; The vehicle-mounted video monitoring method comprises: In response to a trigger signal, determining a target camera corresponding to the trigger signal, the trigger signal comprising at least: a power-on signal; Acquire video data collected by each target camera in the same time period; Grouping the video data to obtain a first video group and a second video group, wherein installation positions of cameras corresponding to the video data in each video group are adjacent in sequence; Performing video stitching processing on the first video group to obtain a first video processing result; Sending the second video group to the second video processing host for the video splicing processing, and obtaining a second video processing result sent by the second video processing host; The video splicing process comprises: sequentially taking the video data in the video group as the data to be spliced ​​and splicing the corresponding preceding splicing data to obtain the video processing result corresponding to the video group, wherein the preceding splicing data is the splicing result of the video data before the video data in the video group; Performing splicing processing on the first video processing result and the second video processing result to obtain panoramic video data; The panoramic video data is transmitted to the display device for display.

2. The vehicle-mounted video monitoring method according to claim 1, characterized in that: The step of sequentially taking the video data in the video group as the data to be spliced ​​and splicing the corresponding preceding spliced ​​data to obtain the video processing result corresponding to the video group includes: According to a preset image frame extraction rule, at least one frame of image is extracted from the data to be spliced ​​and the corresponding preceding spliced ​​data, and each frame of the image corresponds to a timestamp representing the shooting time; Each frame of the image is tested respectively, and the tested image is used as the image to be spliced; Identify a pattern formed by pixels in each frame of the image to be stitched; For every two frames of the images to be spliced ​​with the same timestamp, similarity matching is performed on the graphs formed by the pixel points in the images to be spliced; Splicing every two frames of the images to be spliced ​​that have the same timestamp and whose similarity of the graphics formed by the pixel points reaches a preset threshold, to obtain at least one frame of spliced ​​image; Determining whether there is unjoined video data in the video group; If the unstitched video data does not exist in the video group, generating a video stream based on all the stitched images to obtain a video processing result corresponding to the video group; If the unspliced ​​video data exists in the video group, the spliced ​​image is used as the preceding spliced ​​data corresponding to the next data to be spliced.

3. The vehicle-mounted video monitoring method according to claim 2, characterized in that: The step of performing stitching processing on the first video processing result and the second video processing result to obtain panoramic video data includes: Identify a preset master-slave relationship between the first video processing host and the second video processing host, wherein the backup video processing host in the master-slave relationship is used to perform hot standby redundancy for the main video processing host; In the case where the first video processing host is the master video processing host relative to the second video processing host, the first video processing host respectively obtains stitched images in the first video processing result and the second video processing result, and a timestamp corresponding to each of the stitched images; Identify a pattern formed by pixels in each of the stitched images; Performing similarity matching between a graph formed by pixels in each of the stitched images and a graph formed by pixels in the stitched images with the same timestamp; Stitching the two stitched images with the same timestamp and whose similarity of the graphics formed by the pixel points reaches a preset threshold, to obtain at least one frame of panoramic stitched image; Based on all the panoramic stitched images, a video stream is generated to obtain panoramic video data.

4. The vehicle-mounted video monitoring method according to claim 3, characterized in that: Also includes: In the case where the first video processing host is a backup video processing host relative to the second video processing host, the first video processing host sends the first video processing result and the second video processing result to the second video processing host, so that the second video processing host executes the steps of respectively obtaining the stitched images in the first video processing result and the second video processing result, and the timestamp corresponding to each stitched image.

5. The vehicle-mounted video monitoring method according to claim 2, characterized in that: Also includes: If, among all the images to be stitched, there are no two frames of the images to be stitched that have the same timestamp and whose similarity of the patterns formed by the pixel points reaches a preset threshold, all the images to be stitched are deleted; Extracting an image with the same timestamp from each of the video data in the video group as a target image; Sorting the target images according to the arrangement order of the video data corresponding to the target images in the video group to obtain a target image sequence; Sequentially taking the target images in the target image sequence as target images to be stitched, and performing stitching processing on the corresponding preceding stitched images, to obtain a partial field of view stitched image corresponding to the video group, wherein the preceding stitched image is a stitching result of the target images preceding the target image in the target image sequence; Based on the partial field of view stitching image, a video processing result is generated.

6. The vehicle-mounted video monitoring method according to claim 2, characterized in that: The preset image frame extraction rule at least includes: the number of frames between each two extracted images.

7. The vehicle-mounted video monitoring method according to claim 2, characterized in that: The step of respectively inspecting each frame of the image and using the inspected image as the image to be spliced ​​includes: Checking whether each frame of the image meets a preset condition, wherein the preset condition at least includes: the image does not have distortion or the brightness of the image reaches a preset brightness; If the image meets the preset condition, the image meeting the preset condition is used as the image to be stitched; If the image does not meet the preset condition, image parameters are adjusted for the image meeting the preset condition to obtain the image to be stitched.

8. The vehicle-mounted video monitoring method according to claim 1, characterized in that: In the case where the trigger signal is a turn signal toward the target direction, the method further includes: In response to the trigger signal, the camera that is biased toward the target direction is used as the target camera, and the steps of acquiring the video data collected by each of the target cameras in the same time period and subsequent steps are performed to obtain the panoramic video data of the vehicle in the target direction; The panoramic video data is transmitted to the display device for display so that the driver can monitor the road conditions around the vehicle.

9. A vehicle-mounted video monitoring device, characterized in that: A first video processing host applied to a vehicle-mounted video monitoring system, the vehicle-mounted video monitoring system further comprising: a second video processing host, a display device, and a plurality of cameras installed on the vehicle body, the cameras being used to capture video data around the vehicle, the plurality of cameras being installed at the front of the vehicle and both sides of the vehicle body according to a preset installation interval, and the shooting ranges of two cameras installed at adjacent positions partially overlap; The vehicle-mounted video monitoring device comprises: A signal trigger unit, configured to determine a target camera corresponding to the trigger signal in response to the trigger signal, wherein the trigger signal at least includes: a power-on signal; A data acquisition unit, used to acquire video data collected by each target camera in the same time period; A video grouping unit, used for grouping the video data to obtain a first video group and a second video group, wherein the installation positions of the cameras corresponding to the video data in each video group are adjacent in sequence; A video processing unit, configured to perform video splicing processing on the first video group to obtain a first video processing result; A video result acquisition unit, used for sending the second video group to the second video processing host for the video splicing process, and obtaining a second video processing result sent by the second video processing host; The video splicing process comprises: sequentially taking the video data in the video group as the data to be spliced ​​and splicing the corresponding preceding splicing data to obtain the video processing result corresponding to the video group, wherein the preceding splicing data is the splicing result of the video data before the video data in the video group; A video stitching unit, configured to stitch the first video processing result and the second video processing result to obtain panoramic video data; The panoramic display unit is used to transmit the panoramic video data to the display device for display.

Images