Driver field of vision expansion method, device and electronic equipment based on vehicle event data recorder

Abstract

This application provides a method, device, and electronic device for expanding a driver's field of vision based on a dashcam. The method is applied to a server and includes: acquiring information that a preset vehicle has entered a parking state; receiving a first panoramic video captured by a dashcam installed in the preset vehicle; acquiring information that the preset vehicle has entered a starting state; receiving a second panoramic video captured by the dashcam; comparing the first panoramic video and the second panoramic video to determine whether there are obstacles within a preset range of the preset vehicle; if it is determined that there are obstacles within the preset range, sending the second panoramic video to the vehicle's infotainment screen to display the obstacles. This application can assist in expanding the driver's field of vision during the vehicle's starting phase, thereby helping the driver to detect obstacles in blind spots.

Description

Technical Field

[0001] This application relates to the technical field of driver assistance systems, specifically to a method, device, and electronic device for expanding the driver's field of vision based on a dashcam. Background Technology

[0002] With the increasing number of cars on the road in China, the accident and damage rates of private vehicles are also rising. Major automakers are implementing various measures and technologies in the vehicle design process to ensure vehicle safety during operation and minimize accidents and their consequences. For example, they are equipping vehicles with various driver assistance systems, such as blind spot monitoring, adaptive cruise control, and emergency braking assist systems. These systems provide warnings and assistance to drivers during driving, improving driving safety.

[0003] After a car has been parked for an extended period, the driver should walk counter-clockwise around the vehicle to check for any people, small animals, or other obstacles that might affect safe starting. However, most drivers don't have the habit of doing this beforehand and simply get into the driver's seat. This makes it particularly difficult for drivers to spot people, small animals, or other obstacles in blind spots. Therefore, a method is needed to expand the driver's field of vision during the initial starting phase, helping them to detect obstacles in blind spots. Summary of the Invention

[0004] This application provides a method, device, and electronic device for expanding the driver's field of vision based on a dashcam, which can assist in expanding the driver's field of vision during the vehicle's start-up phase, thereby helping the driver to detect obstacles in the blind spot.

[0005] The first aspect of this application provides a method for expanding the driver's field of vision based on a dashcam, the method being applied to a server, comprising: Obtain information about a preset vehicle entering a parking state; Receives a first panoramic video captured by a dashcam, which is installed in the preset vehicle; Obtain information about the preset vehicle entering the start state; Receive the second panoramic video captured by the vehicle dashcam; The first panoramic video is compared with the second panoramic video to determine whether there are obstacles within the preset range of the preset vehicle; If it is determined that the obstacle exists within the preset range, the second panoramic video is sent to the vehicle's infotainment screen so that the obstacle is displayed on the screen.

[0006] By employing the above technical solution, the server first acquires a first panoramic video of the preset vehicle when it is parked, reflecting the surrounding environment when the preset vehicle just enters the parking phase. When the preset vehicle enters the starting phase, the server acquires a second panoramic video, reflecting the surrounding environment during the starting phase, including blind spots not visible to the driver. By comparing the first and second panoramic videos, and checking for changes, the server determines whether the surrounding environment has changed between the parking and starting phases, thus identifying any obstacles near the preset vehicle. If obstacles are identified, the second panoramic video is sent to the vehicle's screen, allowing the driver to see the real-time surrounding environment, including obstacles in blind spots, thus expanding the driver's field of vision and providing a more comprehensive understanding of the surrounding environment.

[0007] Optionally, comparing the first panoramic video with the second panoramic video to determine whether there are obstacles within a preset range of the preset vehicle specifically includes: The first panoramic video is segmented to obtain multiple first video frames; The second panoramic video is segmented to obtain multiple second video frames; According to a preset method, multiple first video frames are processed to obtain multiple first processed images, and multiple second video frames are processed to obtain multiple second processed images; Identify whether the first target image and the second target image are the same, wherein the first target image is any one of a plurality of first processed images, and the second processed image is any one of a plurality of second processed images; If it is determined that the first target image and the second target image are the same, then the number of second target images that are the same as the first target image among the plurality of second processed images is determined; Determine whether the number of images is greater than or equal to the preset number of images to determine whether there are obstacles within the preset range of the preset vehicle.

[0008] By employing the above technical solution, the server divides the first panoramic video into multiple first video frames and the second panoramic video into multiple second video frames, and then performs preset processing on them to obtain multiple first processed images and second processed images. The server identifies the number of second processed images that are identical to any one of the first processed images and compares this number with a preset threshold. When the number of images is less than the preset threshold, it indicates that multiple first processed images are different from the second processed images, further demonstrating that the first panoramic image is different from the second panoramic image, thus providing a basis for subsequently determining whether there are obstacles within a preset range of the preset vehicle.

[0009] Optionally, identifying whether the first target image and the second target image are the same specifically includes: The first target image is divided into multiple first image blocks of equal size; The second target image is divided into multiple equal-sized second image blocks, wherein the size of the first image block is the same as the size of the second image block; Pixel similarity is calculated for the first image block and the second image block at the same location to obtain the image block similarity. Based on the similarity of multiple image patches, the image similarity between the first target image and the second target image is determined; Determine whether the image similarity is greater than or equal to a preset similarity threshold to determine whether the first target image and the second target image are the same.

[0010] By employing the above technical solution, the first target image and the second target image are each divided into multiple image blocks of equal size. This approach divides a large image into smaller blocks, making pixel similarity calculation more accurate and efficient. It also facilitates subsequent identification of the second image block, which differs from the first image block, to more accurately determine the location of obstacles. The calculated overall image similarity is then compared to a preset similarity threshold. This preset similarity threshold is set based on specific application requirements and actual conditions, serving as a standard for determining whether two images are identical. If the overall image similarity is greater than or equal to the preset similarity threshold, the first target image and the second target image are considered identical; otherwise, they are considered dissimilar.

[0011] Optionally, the dashcam includes a first camera, a second camera, a third camera, and a fourth camera; The first camera is positioned in a first direction of the preset vehicle, the second camera is positioned in a second direction of the preset vehicle, the third camera is positioned in a third direction of the preset vehicle, and the fourth camera is positioned in a fourth direction of the preset vehicle. Any two of the first direction, the second direction, the third direction, and the fourth direction are different. Send a first video capture command to the first camera, so that the first camera captures a first video in the first direction; Send a second video capture command to the second camera, so that the second camera captures a second video in the second direction; Send a third video capture command to the third camera, so that the third camera captures the third video in the third direction; A fourth video capture command is sent to the fourth camera so that the fourth camera captures a fourth video in the fourth direction.

[0012] By employing the aforementioned technical solution and setting up four cameras to cover different directions of the vehicle, multi-directional panoramic monitoring of the vehicle's surroundings can be achieved. This layout allows the server to obtain 360-degree field-of-view information about the vehicle's surroundings, including the front, rear, left, and right sides. The multiple cameras capture images from more different angles, including blind spots and areas that are normally difficult for the driver to directly observe. By acquiring this multi-directional video, the server can provide a more comprehensive and detailed view of the area around the vehicle, helping to expand the driver's field of vision and thus assisting the driver in identifying obstacles in blind spots.

[0013] Optionally, receiving the first panoramic video captured by the dashcam specifically includes: Receive the first video sent by the first camera, the second video sent by the second camera, the third video sent by the third camera, and the fourth video sent by the fourth camera; The first panoramic video is obtained by stitching together the first video, the second video, the third video, and the fourth video according to the first direction, the second direction, the third direction, and the fourth direction.

[0014] By employing the aforementioned technical solution, and receiving videos from the first, second, third, and fourth cameras, visual information from different directions around the vehicle can be obtained. These videos are then stitched together to form a first panoramic video, presenting a 360-degree panoramic view of the vehicle's surroundings and providing more comprehensive and detailed information about the vehicle's environment. Stitching together videos from multiple directions yields a real-time panoramic video, displaying the overall situation of the vehicle's surroundings. This real-time monitoring function allows the driver to observe the vehicle's surroundings in real time, including blind spots, enabling timely detection of obstacles or dangerous situations and improving driving safety.

[0015] Optionally, after determining that an obstacle exists within the preset range, and then sending the second panoramic video to the vehicle's infotainment screen to display the obstacle on the screen, the method further includes: Obtain the position of a second image patch that is different from the first image patch, and determine the position of the obstacle in the second panoramic video; Based on the location, the obstacle is pre-marked in the second panoramic video.

[0016] By adopting the above technical solution, once an obstacle is detected within a preset range, the server immediately sends a second panoramic video to the in-vehicle screen of the preset vehicle. This real-time warning function allows the driver to see obstacles around the vehicle in a timely manner, especially obstacles in blind spots, improving the driver's alertness and helping the driver take timely evasive or stopping measures to ensure driving safety.

[0017] Optionally, the step of processing multiple first video frames according to a preset method to obtain multiple first processed images, and processing multiple second video frames to obtain multiple second processed images, specifically includes: Each of the first video frames is enhanced to obtain multiple first enhanced images; Denoising is performed on each of the first enhanced images to obtain multiple first denoised images; Each of the first denoised images is desaturated to obtain multiple first processed images; Each of the second video frames is enhanced to obtain multiple second enhanced images; Denoising is performed on each of the second enhanced images to obtain multiple second denoised images; Each of the second denoised images is desaturated to obtain multiple second processed images.

[0018] By employing the above technical solutions to enhance multiple first and second video frames, the brightness, contrast, and color saturation of the image can be improved, making the image clearer and brighter. During video acquisition, environmental conditions or camera limitations may cause noise or graininess in the video, affecting image quality. Noise reduction processing on multiple enhanced first and second images can reduce noise, making the image smoother and clearer. Desaturation processing converts color images to grayscale or binary images, simplifying image data processing and accelerating image analysis. Through these processing steps, image quality can be significantly improved, noise and interference reduced, and images made clearer and more reliable, providing a better data foundation for subsequent image recognition and analysis.

[0019] A second aspect of this application provides a driver's field of vision expansion device based on a dashcam. The device is a server, comprising a first acquisition module, a first receiving module, a second acquisition module, a second receiving module, a processing module, and an output module, wherein: The first acquisition module is used to acquire information about a preset vehicle entering a parking state; The first receiving module is used to receive the first panoramic video captured by the dashcam, which is installed in the preset vehicle; The second acquisition module is used to acquire information about the preset vehicle entering the start state; The second receiving module is used to receive the second panoramic video captured by the dashcam; The processing module is used to compare the first panoramic video with the second panoramic video to determine whether there are obstacles within the preset range of the preset vehicle; The output module is configured to send the second panoramic video to the vehicle's infotainment screen if it is determined that an obstacle exists within the preset range, so that the vehicle's infotainment screen displays the obstacle.

[0020] A third aspect of this application provides an electronic device including a processor, a memory, a user interface, and a network interface, wherein the memory is used to store instructions, the user interface and the network interface are both used to communicate with other devices, and the processor is used to execute the instructions stored in the memory to cause the electronic device to perform the method as described in any of the foregoing.

[0021] A fourth aspect of this application provides a computer-readable storage medium storing instructions that, when executed, perform the method described in any of the preceding descriptions.

[0022] In summary, one or more technical solutions provided in the embodiments of this application have at least the following technical effects or advantages: 1. The server first acquires a first panoramic video of the preset vehicle when it is parked, reflecting the surrounding environment when the vehicle first enters the parking phase. When the preset vehicle enters the starting phase, the server acquires a second panoramic video, reflecting the surrounding environment during the starting phase, including blind spots not visible to the driver. By comparing the first and second panoramic videos, and checking for changes, the server determines whether the surrounding environment has changed between the parking and starting phases, thus identifying any obstacles nearby. If obstacles are found, the second panoramic video is sent to the vehicle's screen, allowing the driver to see the real-time surroundings, including obstacles in blind spots, thus expanding the driver's field of vision and providing a more comprehensive understanding of the vehicle's surroundings.

[0023] 2. The server divides the first panoramic video into multiple first video frames and the second panoramic video into multiple second video frames, and then performs preset processing on them to obtain multiple first processed images and second processed images. The server identifies the number of second processed images that are identical to any one of the first processed images and compares this number with a preset threshold. If the number of images is less than the preset threshold, it indicates that multiple first processed images are different from the second processed images, further demonstrating that the first panoramic image is different from the second panoramic image, thus providing a basis for subsequent determination of whether there are obstacles within a preset range of the preset vehicle.

[0024] 3. By setting up four cameras, each covering different directions of the vehicle, multi-directional panoramic monitoring of the vehicle's surroundings can be achieved. This layout allows the server to obtain 360-degree field-of-view information about the vehicle's surroundings, including the front, rear, left, and right sides. The multiple cameras capture images from more different angles, including blind spots and areas that are normally difficult for the driver to see directly. By acquiring this multi-directional video, the server can provide a more comprehensive and detailed view of the area around the vehicle, helping to expand the driver's field of vision and thus assisting the driver in detecting obstacles in blind spots. Attached Figure Description

[0025] Figure 1 This is a flowchart illustrating a method for expanding the driver's field of vision based on a dashcam, as disclosed in an embodiment of this application. Figure 2 A schematic diagram of the shooting range of a dashcam disclosed in an embodiment of this application; Figure 3This is a schematic diagram of a driver's field of vision expansion device based on a driving recorder disclosed in an embodiment of this application; Figure 4 This is a schematic diagram of the structure of an electronic device disclosed in an embodiment of this application.

[0026] Explanation of reference numerals in the attached drawings: 301, First acquisition module; 302, First receiving module; 303, Second acquisition module; 304, Second receiving module; 305, Processing module; 306, Output module; 401, Processor; 402, Communication bus; 403, User interface; 404, Network interface; 405, Memory. Detailed Implementation

[0027] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments.

[0028] In the description of the embodiments of this application, the words "for example" or "for instance" are used to indicate examples, illustrations, or explanations. Any embodiment or design that is described as "for example" or "for instance" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design options. Rather, the use of the words "for example" or "for instance" is intended to present the relevant concepts in a specific manner.

[0029] In the description of the embodiments of this application, the term "multiple" means two or more. For example, multiple systems means two or more systems, and multiple screen terminals means two or more screen terminals. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the indicated technical features. Thus, a feature defined with "first" or "second" may explicitly or implicitly include one or more of that feature. The terms "comprising," "including," "having," and variations thereof all mean "including but not limited to," unless otherwise specifically emphasized.

[0030] With the increasing number of cars on the road in China, the accident and damage rates of private vehicles are also rising. Major automakers are implementing various measures and technologies in the vehicle design process to ensure vehicle safety during operation and minimize accidents and their consequences. For example, they are equipping vehicles with various driver assistance systems, such as blind spot monitoring, adaptive cruise control, and emergency braking assist systems. These systems provide warnings and assistance to drivers during driving, improving driving safety.

[0031] After a car has been parked for an extended period, the driver should walk counter-clockwise around the vehicle to check for any people, small animals, or other obstacles that might affect safe starting. However, most drivers don't have the habit of doing this beforehand and simply get into the driver's seat. This makes it particularly difficult for drivers to spot people, small animals, or other obstacles in blind spots. Therefore, a method is needed to expand the driver's field of vision during the initial starting phase, helping them to detect obstacles in blind spots.

[0032] This embodiment discloses a method for expanding the driver's field of vision based on a dashcam, referring to... Figure 1 This includes the following steps S110-S160: S110, obtain information about the preset vehicle entering a parking state.

[0033] The driver's field of vision expansion method based on a dashcam disclosed in this application is applied to a server installed in a pre-set vehicle. The server, i.e., the vehicle infotainment system, is the central control unit inside the car, responsible for integrating and managing in-vehicle entertainment, information, navigation, and vehicle control functions. Through the server, users can operate the vehicle's audio, video, and navigation systems and interact with the system via touchscreen, knobs, voice recognition, etc. It can also connect to mobile phones, the internet, and other external devices to provide real-time information. Simultaneously, the vehicle infotainment system can also monitor vehicle status, display vehicle diagnostic information, and provide driver assistance functions.

[0034] A vehicle's ignition switch typically has multiple positions, including off, ACC (auxiliary power), ON (ignition), and START (engine start). By reading the ignition switch's status, the server can determine whether the vehicle is off and, consequently, whether the vehicle is in a preset parking state.

[0035] S120 receives the first panoramic video captured by the dashcam.

[0036] This application discloses a dashcam, which is installed in a preset vehicle. The dashcam includes four cameras: a first camera, a second camera, a third camera, and a fourth camera. The four cameras are respectively positioned at four different locations within the preset vehicle. The first, second, third, and fourth cameras preferably all employ wide-angle lenses. Figure 2The four cameras are positioned to ensure their shooting range is within a preset range, covering the entire circumference of the vehicle. Specifically, the first camera is positioned in a first direction relative to the vehicle, the second in a second direction, the third in a third direction, and the fourth in a fourth direction. No two of the first, second, third, and fourth directions are the same. Preferably, in this embodiment, the first camera is positioned directly in front of the vehicle, mounted on the windshield. The second camera is positioned directly behind the vehicle, mounted on the rear windshield. The third camera is positioned directly to the left of the vehicle, below the left rearview mirror. The fourth camera is positioned directly to the right of the vehicle, below the right rearview mirror.

[0037] By setting up four cameras, each covering different directions of the vehicle, multi-directional panoramic monitoring of the vehicle's surroundings can be achieved. This layout allows the server to obtain 360-degree field-of-view information about the vehicle's surroundings, including the front, rear, left, and right sides. The multiple cameras capture images from more different angles, including blind spots and areas that are normally difficult for the driver to see directly. By acquiring this multi-directional video, the server can provide a more comprehensive and detailed view of the area around the vehicle, helping to expand the driver's field of vision and thus assisting the driver in detecting obstacles in blind spots.

[0038] After the server determines that the preset vehicle is in a parked state, it sends a first video capture command to the first camera, causing the first camera to capture a first video in a first direction. It then sends a second video capture command to the second camera, causing the second camera to capture a second video in a second direction. Finally, it sends a third video capture command to the third camera, causing the third camera to capture a third video in a third direction. Finally, it sends a fourth video capture command to the fourth camera, causing the fourth camera to capture a fourth video in a fourth direction.

[0039] The first camera sends the captured first video to the server, the second camera sends the captured second video to the server, the third camera sends the captured third video to the server, and the fourth camera sends the captured fourth video to the server. The server receives the first video from the first camera, the second video from the second camera, the third video from the third camera, and the fourth video from the fourth camera.

[0040] Next, the first, second, third, and fourth videos undergo preprocessing, including noise reduction, color correction, and resizing, to ensure the accuracy and stability of subsequent stitching. Then, computer vision techniques are used to extract feature points from the video frames in the four directions, and feature point matching is used to find the correspondences between them. These feature points can be corner points, edges, textures, etc. Using the results of feature point matching, the video frames in the four directions are registered, that is, their positions and angles are aligned so that they can be correctly stitched together. Based on the registered video frames, they are stitched together horizontally to form a continuous panoramic image. In other words, the first, second, third, and fourth videos are stitched together according to the first, second, third, and fourth directions to obtain the first panoramic video. During the stitching process, it may be necessary to consider the overlapping parts between different viewpoints and adopt appropriate fusion methods to avoid stitching imperfections.

[0041] By receiving videos from the first, second, third, and fourth cameras, visual information from different directions around the vehicle can be obtained. These videos are then stitched together to form a first panoramic video, presenting a 360-degree panoramic view of the vehicle's surroundings and providing more comprehensive and detailed information about the environment. By stitching together videos from multiple directions, a real-time panoramic video is obtained, displaying the overall situation of the vehicle's surroundings. This real-time monitoring function allows the driver to observe the vehicle's surroundings in real time, including blind spots, enabling timely detection of obstacles or hazards and improving driving safety.

[0042] S130, obtain information about the preset vehicle entering the start state.

[0043] The server monitors the vehicle's power status using sensors pre-set on the vehicle. For example, it determines whether the vehicle is running by detecting the battery voltage or current. When the vehicle's power is on, the battery voltage or current will change accordingly. By monitoring these changes, the server can determine whether the vehicle is running.

[0044] S140 receives the second panoramic video captured by the dashcam.

[0045] Referring to step S120, after determining that the preset vehicle has entered the start state, the server sends a first video acquisition command to the first camera again, so that the first camera acquires the first video in the first direction. A second video acquisition command is sent to the second camera, so that the second camera acquires the second video in the second direction. A third video acquisition command is sent to the third camera, so that the third camera acquires the third video in the third direction. A fourth video acquisition command is sent to the fourth camera, so that the fourth camera acquires the fourth video in the fourth direction.

[0046] After the first, second, third, and fourth cameras have completed capturing data, the first camera sends its first captured video to the server, the second camera sends its second captured video to the server, the third camera sends its third captured video to the server, and the fourth camera sends its fourth captured video to the server. The server receives the first video from the first camera, the second video from the second camera, the third video from the third camera, and the fourth video from the fourth camera. After preprocessing the first, second, third, and fourth videos, including denoising, color correction, and size adjustment, the server stitches them together to obtain the second panoramic video.

[0047] S150: Compare the first panoramic video with the second panoramic video to determine whether there are obstacles within the preset range of the preset vehicle.

[0048] Specifically, the server first segments the first panoramic video into multiple first video frames, and then segments the second panoramic video into multiple second video frames. Next, according to a preset method, the multiple first video frames are processed to obtain multiple first processed images, and the multiple second video frames are processed to obtain multiple second processed images. Specifically, the first video frames are first enhanced, using a series of operations to improve image quality and visualization. Common enhancement techniques include adjusting contrast, brightness, and color balance, and image processing libraries such as OpenCV can be used to implement image enhancement. The enhanced first video frames yield the first enhanced image. Then, the first enhanced image undergoes noise reduction to reduce noise or unnecessary details. Common noise reduction methods include median filtering and Gaussian filtering. These filters help smooth the image and remove noise, resulting in the first denoised image. Finally, the first denoised image is desaturated, converting it to grayscale and removing color information from the color image, retaining only brightness information. This simplifies the image and accelerates subsequent processing, ultimately yielding the first processed image. The processing procedure for the second video frame is the same as that for the first video frame. Each second video frame is sequentially enhanced to obtain multiple enhanced second images. Each enhanced second image is then denoised to obtain multiple denoised second images. Finally, each denoised second image is desaturated to obtain multiple processed second images.

[0049] Enhancing multiple first and second video frames can improve image brightness, contrast, and color saturation, making the image clearer and brighter. During video acquisition, environmental conditions or camera limitations may introduce noise or graininess, affecting image quality. Noise reduction processing on multiple enhanced first and second images reduces noise, resulting in a smoother and clearer image. Desaturation converts color images to grayscale or binary images, simplifying image data processing and accelerating image analysis. These processing steps significantly improve image quality, reduce noise and interference, and make images clearer and more reliable, providing a better data foundation for subsequent image recognition and analysis.

[0050] After obtaining multiple first-processed images and multiple second-processed images, each of the first-processed images and any one of the second-processed images is then identified to determine whether they are the same. For example, consider any one of the first-processed images, a first target image, and any one of the second-processed images, a second target image.

[0051] The server segments the first target image into multiple equal-sized first image blocks, for example, each first image block is 10×20mm. Similarly, the second target image is segmented into multiple equal-sized second image blocks, with the second image blocks having the same size as the first image blocks. During segmentation, a fixed-size image block can be set, or different image block sizes can be set according to the specific application scenario and algorithm requirements. For example, if the first image block is 10×20mm, then the second image block will also be 10×20mm. Next, pixel similarity is calculated for the first and second image blocks at the same location. Mean Squared Error (MSE) can be used to calculate the average difference between the pixels of the two image blocks; a smaller value indicates greater image similarity. Structural Similarity (SSIM) can also be used, comprehensively considering brightness, contrast, and structural similarity; the value ranges from -1 to 1, with values ​​closer to 1 indicating greater image similarity. Peak Signal-to-Noise Ratio (PSNR) can also be used to calculate the signal-to-noise ratio between the two image blocks; a larger value indicates greater image similarity. These methods can compare pixel differences between two image patches. An appropriate similarity calculation method can be selected based on actual needs, or multiple methods can be combined to comprehensively analyze the image patch similarity between the first and second image patches. Based on the similarity values ​​of multiple image patches, the overall image similarity between the first and second target images is determined. The average, weighted average, or maximum value of the image patch similarities can be calculated to obtain the overall image similarity.

[0052] The calculated overall image similarity is compared with a preset similarity threshold. The preset similarity threshold is set according to specific application requirements and actual conditions, and is used to determine whether two images are identical; this embodiment does not impose specific limitations. If the image similarity is greater than or equal to the preset similarity threshold, the first target image and the second target image are determined to be identical; otherwise, the two images are determined to be different.

[0053] The first and second target images are each divided into multiple equally sized image blocks. This approach segments the large image into smaller blocks, making pixel similarity calculation more accurate and efficient. It also facilitates subsequent identification of obstacle locations by recognizing second image blocks that differ from the first image block. The calculated overall image similarity is compared to a preset similarity threshold. This threshold, set based on specific application requirements and practical considerations, serves as the standard for determining whether two images are identical. If the overall image similarity is greater than or equal to the preset similarity threshold, the first and second target images are considered identical; otherwise, they are considered dissimilar.

[0054] When it is determined that among multiple second-processed images, there is a second target image identical to the first target image, the server then counts the number of images of the same second target image. Finally, it determines whether the number of images is greater than or equal to a preset number of images to determine whether there are obstacles within a preset range of the preset vehicle. When the number of second-processed images identical to the first-processed image is greater than or equal to the preset number of images, it indicates that the environment near the preset vehicle before parking is the same as the environment near the preset vehicle when it starts, that is, during parking, no person, animal, or other small obstacle approaches the vehicle. When the number of second-processed images identical to the first-processed image is less than the preset number of images, it indicates that during the parking phase, a person, animal, or other small obstacle may approach the vehicle and remain near the preset vehicle, causing the multiple first-processed images obtained by the server at startup to be different from the second-processed images obtained by the server at parking.

[0055] The server segments the first panoramic video into multiple first video frames and the second panoramic video into multiple second video frames, and then performs preset processing on them to obtain multiple first processed images and second processed images. The server identifies the number of second processed images that are identical to any one of the first processed images and compares this number with a preset threshold. If the number of images is less than the preset threshold, it indicates that multiple first processed images are different from the second processed images, further demonstrating that the first panoramic image and the second panoramic image are different, thus providing a basis for subsequent determination of whether there are obstacles within a preset range of the preset vehicle.

[0056] S160, if it is determined that there is an obstacle within the preset range, a second panoramic video is sent to the vehicle screen of the preset vehicle so that the vehicle screen displays the obstacle.

[0057] When the number of second processed images identical to the first processed image is less than the preset number of images, it indicates that an obstacle exists within the preset range of the preset vehicle. The server then sends a second panoramic video to the vehicle's infotainment screen to display the obstacle. Based on the second image block calculated by the server in the above steps, which is different from the first image block, the server obtains the specific location of the second image block in the second panoramic video. Then, based on the specific location, the detected obstacle is marked in the second panoramic video. The location of the second image block can be marked using a rectangle or other shapes on the image, or text or other markings can be added to the image to indicate the type or attribute of the obstacle in the second image block. After determining that an obstacle exists within the preset range, the server immediately sends the second panoramic video to the vehicle's infotainment screen. This real-time warning function allows the driver to see obstacles around the vehicle in a timely manner, especially obstacles in blind spots, improving the driver's alertness and helping the driver take timely evasive or stopping measures to ensure driving safety.

[0058] By employing the above technical solution, the server first acquires a first panoramic video of the preset vehicle when it is parked, reflecting the surrounding environment when the preset vehicle just enters the parking phase. When the preset vehicle enters the starting phase, the server acquires a second panoramic video, reflecting the surrounding environment during the starting phase, including blind spots not visible to the driver. By comparing the first and second panoramic videos, and checking for changes, the server determines whether the surrounding environment has changed between the parking and starting phases, thus identifying any obstacles near the preset vehicle. If obstacles are identified, the second panoramic video is sent to the vehicle's screen, allowing the driver to see the real-time surrounding environment, including obstacles in blind spots, thus expanding the driver's field of vision and providing a more comprehensive understanding of the surrounding environment.

[0059] This embodiment also discloses a driver's field of vision expansion device based on a dashcam, wherein the device is a server, as described above. Figure 3 It includes a first acquisition module 301, a first receiving module 302, a second acquisition module 303, a second receiving module 304, a processing module 305, and an output module 306, wherein: The first acquisition module 301 is used to acquire information about a preset vehicle entering a parking state.

[0060] The first receiving module 302 is used to receive the first panoramic video captured by the dashcam, which is installed in a preset vehicle.

[0061] The second acquisition module 303 is used to acquire information about a preset vehicle entering the start state.

[0062] The second receiving module 304 is used to receive the second panoramic video captured by the dashcam.

[0063] The processing module 305 is used to compare the first panoramic video with the second panoramic video to determine whether there are obstacles within the preset range of the preset vehicle.

[0064] The output module 306 is used to send a second panoramic video to the vehicle screen of a preset vehicle if it is determined that there is an obstacle within a preset range, so that the vehicle screen displays the obstacle.

[0065] In one possible implementation, the processing module 305 is used to segment the first panoramic video to obtain multiple first video frames.

[0066] The processing module 305 is used to segment the second panoramic video to obtain multiple second video frames.

[0067] The processing module 305 is used to process multiple first video frames according to a preset method to obtain multiple first processed images, and to process multiple second video frames to obtain multiple second processed images.

[0068] The processing module 305 is used to identify whether the first target image and the second target image are the same, wherein the first target image is any one of a plurality of first processed images, and the second processed image is any one of a plurality of second processed images.

[0069] The processing module 305 is used to determine the number of second target images that are the same as the first target image among a plurality of second processed images if it is determined that the first target image and the second target image are the same.

[0070] The processing module 305 is used to determine whether the number of images is greater than or equal to the preset number of images, so as to determine whether there are obstacles within the preset range of the preset vehicle.

[0071] In one possible implementation, the processing module 305 is used to divide the first target image into multiple first image blocks of equal size.

[0072] The processing module 305 is used to divide the second target image into multiple second image blocks of equal size, wherein the size of the first image block is the same as the size of the second image block.

[0073] The processing module 305 is used to calculate the pixel similarity between the first image block and the second image block at the same location to obtain the image block similarity.

[0074] The processing module 305 is used to determine the image similarity between the first target image and the second target image based on the similarity of multiple image patches.

[0075] The processing module 305 is used to determine whether the image similarity is greater than or equal to a preset similarity threshold, so as to determine whether the first target image and the second target image are the same.

[0076] In one possible implementation, the dashcam includes a first camera, a second camera, a third camera, and a fourth camera.

[0077] The first camera is positioned in the first direction of the preset vehicle, the second camera is positioned in the second direction of the preset vehicle, the third camera is positioned in the third direction of the preset vehicle, and the fourth camera is positioned in the fourth direction of the preset vehicle. No two of the first, second, third, and fourth directions are the same.

[0078] The output module 306 is used to send a first video acquisition command to the first camera so that the first camera can acquire a first video in a first direction.

[0079] The output module 306 is used to send a second video acquisition command to the second camera so that the second camera can acquire a second video in a second direction.

[0080] The output module 306 is used to send a third video acquisition command to the third camera so that the third camera can acquire a third video in a third direction.

[0081] The output module 306 is used to send a fourth video acquisition command to the fourth camera so that the fourth camera can acquire a fourth video from a fourth direction.

[0082] In one possible implementation, the first receiving module 302 is used to receive a first video sent by a first camera, a second video sent by a second camera, a third video sent by a third camera, and a fourth video sent by a fourth camera.

[0083] The processing module 305 is used to stitch together the first video, the second video, the third video, and the fourth video according to the first direction, the second direction, the third direction, and the fourth direction to obtain the first panoramic video.

[0084] In one possible implementation, the first acquisition module 301 is used to acquire the position of a second image block that is different from the first image block, and to determine the position of the obstacle in the second panoramic video.

[0085] The processing module 305 is used to pre-mark obstacles in the second panoramic video according to the location.

[0086] In one possible implementation, the processing module 305 is used to perform enhancement processing on each of the first video frames to obtain a plurality of first enhanced images.

[0087] The processing module 305 is used to perform noise reduction processing on each of the first enhanced images to obtain multiple first denoised images.

[0088] The processing module 305 is used to desaturate each of the first noise-reduced images to obtain multiple first processed images.

[0089] The processing module 305 is used to perform enhancement processing on each second video frame to obtain multiple second enhanced images.

[0090] The processing module 305 is used to perform noise reduction processing on each of the second enhanced images to obtain multiple second noise-reduced images.

[0091] The processing module 305 is used to desaturate each of the second noise-reduced images to obtain multiple second processed images.

[0092] It should be noted that the above embodiments of the apparatus are only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0093] This embodiment also discloses an electronic device, as shown in the reference. Figure 4 The electronic device may include: at least one processor 401, at least one communication bus 402, user interface 403, network interface 404, and at least one memory 405.

[0094] The communication bus 402 is used to enable communication between these components.

[0095] The user interface 403 may include a display screen and a camera. Optionally, the user interface 403 may also include a standard wired interface and a wireless interface.

[0096] The network interface 404 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface).

[0097] The processor 401 may include one or more processing cores. The processor 401 connects to various parts of the server using various interfaces and lines, and performs various server functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in memory 405, and by calling data stored in memory 405. Optionally, the processor 401 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor 401 may integrate one or a combination of several of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content required for display; and the modem handles wireless communication. It is understood that the modem may also be implemented as a separate chip without being integrated into the processor 401.

[0098] The memory 405 may include random access memory (RAM) or read-only memory. Optionally, the memory 405 may include a non-transitory computer-readable storage medium. The memory 405 may be used to store instructions, programs, code, code sets, or instruction sets. The memory 405 may include a program storage area and a data storage area. The program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch function, sound playback function, image playback function, etc.), instructions for implementing the above-described method embodiments, etc.; the data storage area may store data involved in the above-described method embodiments, etc. Optionally, the memory 405 may also be at least one storage device located remotely from the aforementioned processor 401. As shown in the figure, the memory 405, as a computer storage medium, may include an operating system, a network communication module, a user interface 403 module, and an application program for a driver's field of vision expansion method based on a dashcam.

[0099] exist Figure 4In the electronic device shown, the user interface 403 is mainly used to provide an input interface for the user and to obtain the user input data; while the processor 401 can be used to call the application program stored in the memory 405 for the driver's field of vision expansion method based on the dashcam. When executed by one or more processors 401, the electronic device performs one or more methods as described in the above embodiments.

[0100] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0101] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0102] In the several embodiments provided in this application, it should be understood that the disclosed apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the shown or discussed mutual couplings or direct couplings or communication connections may be through some service interfaces; indirect couplings or communication connections between apparatuses or units may be electrical or other forms.

[0103] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0104] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0105] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage device 405. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage device 405 and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage device 405 includes various media capable of storing program code, such as a USB flash drive, external hard drive, magnetic disk, or optical disk.

[0106] The foregoing description is merely an exemplary embodiment of this disclosure and should not be construed as limiting the scope of this disclosure. Any equivalent changes and modifications made in accordance with the teachings of this disclosure shall still fall within the scope of this disclosure. Those skilled in the art will readily conceive of other embodiments of this disclosure upon considering the specification and the disclosure of practical truth. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not described in this disclosure. The specification and embodiments are considered exemplary only, and the scope and spirit of this disclosure are defined by the claims.

Claims

1. A method for expanding a driver's field of view based on a dashcam, characterized by, The method is applied to a server and includes: Obtain information about a preset vehicle entering a parking state; Receives a first panoramic video captured by a dashcam, which is installed in the preset vehicle; Obtain information about the preset vehicle entering the start state; Receive the second panoramic video captured by the vehicle dashcam; The first panoramic video is compared with the second panoramic video to determine whether there are obstacles within a preset range of the preset vehicle; the comparison of the first panoramic video with the second panoramic video to determine whether there are obstacles within a preset range of the preset vehicle specifically includes: segmenting the first panoramic video to obtain multiple first video frames; segmenting the second panoramic video to obtain multiple second video frames; According to a preset method, multiple first video frames are processed to obtain multiple first processed images, and multiple second video frames are processed to obtain multiple second processed images; it is then determined whether a first target image and a second target image are the same, where the first target image is any one of the multiple first processed images, and the second processed image is any one of the multiple second processed images; if it is determined that the first target image and the second target image are the same, the number of second target images that are the same as the first target image among the multiple second processed images is determined; it is then determined whether the number of images is greater than or equal to a preset number of images to determine whether there is an obstacle within a preset range of the preset vehicle; the identification of whether the first target image and the second target image are the same specifically includes: dividing the first target image into multiple first image blocks of equal size; dividing the second target image into multiple second image blocks of equal size, where the size of the first image block is the same as the size of the second image block; performing pixel similarity calculation on the first image block and the second image block at the same position to obtain image block similarity; determining the image similarity between the first target image and the second target image based on the multiple image block similarities; and determining whether the image similarity is greater than or equal to a preset similarity threshold to determine whether the first target image and the second target image are the same; If it is determined that the obstacle exists within the preset range, the second panoramic video is sent to the vehicle's infotainment screen so that the obstacle is displayed on the screen.

2. The method of claim 1, wherein the method is based on a dashcam. The method further includes: The dashcam includes a first camera, a second camera, a third camera, and a fourth camera; The first camera is positioned in a first direction of the preset vehicle, the second camera is positioned in a second direction of the preset vehicle, the third camera is positioned in a third direction of the preset vehicle, and the fourth camera is positioned in a fourth direction of the preset vehicle. Any two of the first direction, the second direction, the third direction, and the fourth direction are different. Send a first video capture command to the first camera, so that the first camera captures a first video in the first direction; Send a second video capture command to the second camera, so that the second camera captures a second video in the second direction; Send a third video capture command to the third camera, so that the third camera captures the third video in the third direction; A fourth video capture command is sent to the fourth camera so that the fourth camera captures a fourth video in the fourth direction.

3. The method of claim 2, wherein the method is based on a dashcam. The first panoramic video captured by the dashcam specifically includes: Receive the first video sent by the first camera, the second video sent by the second camera, the third video sent by the third camera, and the fourth video sent by the fourth camera; The first panoramic video is obtained by stitching together the first video, the second video, the third video, and the fourth video according to the first direction, the second direction, the third direction, and the fourth direction.

4. The dashcam-based method of expanding a driver's field of view according to claim 1, wherein, After determining that an obstacle exists within the preset range, and then sending the second panoramic video to the vehicle's infotainment screen to display the obstacle on the screen, the method further includes: Obtain the position of a second image patch that is different from the first image patch, and determine the position of the obstacle in the second panoramic video; Based on the location, the obstacle is pre-marked in the second panoramic video.

5. The dashcam-based method of expanding a driver's field of view according to claim 1, wherein, The process of processing multiple first video frames according to a preset method to obtain multiple first processed images, and processing multiple second video frames to obtain multiple second processed images, specifically includes: Each of the first video frames is enhanced to obtain multiple first enhanced images; Denoising is performed on each of the first enhanced images to obtain multiple first denoised images; Each of the first denoised images is desaturated to obtain multiple first processed images; Each of the second video frames is enhanced to obtain multiple second enhanced images; Denoising is performed on each of the second enhanced images to obtain multiple second denoised images; Each of the second denoised images is desaturated to obtain multiple second processed images.

6. A driver's field of view widening device based on a dashcam, characterized by, The device is a server, comprising a first acquisition module (301), a first receiving module (302), a second acquisition module (303), a second receiving module (304), a processing module (305), and an output module (306), wherein: The first acquisition module (301) is used to acquire information about a preset vehicle entering a parking state; The first receiving module (302) is used to receive the first panoramic video collected by the dashcam, which is installed in the preset vehicle; The second acquisition module (303) is used to acquire information about the preset vehicle entering the start state; The second receiving module (304) is used to receive the second panoramic video captured by the dashcam; The processing module (305) is used to compare the first panoramic video with the second panoramic video to determine whether there are obstacles within a preset range of the preset vehicle; the comparison of the first panoramic video with the second panoramic video to determine whether there are obstacles within the preset range of the preset vehicle specifically includes: segmenting the first panoramic video to obtain multiple first video frames; segmenting the second panoramic video to obtain multiple second video frames; processing the multiple first video frames according to a preset method to obtain multiple first processed images; processing the multiple second video frames to obtain multiple second processed images; identifying whether a first target image and a second target image are the same, wherein the first target image is any one of the multiple first processed images, and the second processed image is any one of the multiple second processed images; if it is determined that the first target image and the second target image are the same, the comparison is performed on the second target image. If the target images are the same, the number of second target images that are the same as the first target image among multiple second processed images is determined; it is determined whether the number of images is greater than or equal to a preset number of images to determine whether there is an obstacle within a preset range of the preset vehicle; the identification of whether the first target image and the second target image are the same specifically includes: dividing the first target image into multiple first image blocks of equal size; dividing the second target image into multiple second image blocks of equal size, wherein the size of the first image block is the same as the size of the second image block; performing pixel similarity calculation on the first image block and the second image block at the same position to obtain image block similarity; determining the image similarity between the first target image and the second target image based on multiple image block similarities; determining whether the image similarity is greater than or equal to a preset similarity threshold to determine whether the first target image and the second target image are the same; The output module (306) is used to send the second panoramic video to the vehicle screen of the preset vehicle if it is determined that the obstacle exists within the preset range, so that the vehicle screen displays the obstacle.

7. An electronic device, comprising: The device includes a processor (401), a memory (405), a user interface (403), and a network interface (404). The memory (405) is used to store instructions. The user interface (403) and the network interface (404) are both used to communicate with other devices. The processor (401) is used to execute the instructions stored in the memory (405) to cause the electronic device to perform the method as described in any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed, perform the method as described in any one of claims 1-5.

Images