Panoramic system view control method

Abstract

The invention provides a panoramic system view control method, which comprises the following steps of: 1, initializing a system, and displaying a first view on a first target display device; 2, obtaining a sight focus of a driver; and step 3, judging whether the sight focus is within a preset range, and if so, displaying a second view corresponding to the sight focus on a first target display device. According to the invention, the driver can conveniently acquire the environment information around the vehicle, and the driving safety is improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a panoramic system view control method. Background Art

[0002] With the advancement of electronic technology, panoramic view systems have become a mainstream feature in vehicles of all levels, particularly new energy vehicles. These systems stitch together images from multiple body cameras to create a bird's-eye view, allowing drivers to instantly grasp the vehicle's surroundings and improving driving safety.

[0003] However, existing in-vehicle panoramic imaging systems usually display the vehicle's surrounding environment in a fixed perspective or preset mode and cannot actively respond to user needs. When switching between different views, the driver needs to manually switch between multiple display screens to display the desired angle, which causes distraction and affects driving safety. Summary of the Invention

[0004] The present invention provides a panoramic system view control method, which aims to solve the defects in the existing technology, facilitate the driver to obtain environmental information around the vehicle, and improve driving safety.

[0005] In order to achieve the above object, the technical solution adopted by the present invention is: The present invention provides a panoramic system view control method, comprising: Step 1: Initialize the system and display a first view on a first target display device; Step 2: Obtain the driver's sight focus; Step 3: Determine whether the sight focus is within a preset range, and if so, display a second view corresponding to the sight focus on the first target display device.

[0006] Furthermore, after step 3, the method further includes: Step 4: Detect whether there is a target obstacle in the second view, and if so, mark the target obstacle; Step 5: Synchronously updating the target obstacle information to the second target display device; Step 6: Determine whether the sight focus deviates from the current driving direction for more than a preset time, and if so, issue an alarm in a preset manner.

[0007] Specifically, the step 2 includes: Step 201: Acquire the driver's facial image in real time; Step 202: Recognize the facial image to obtain the driver's sight focus; Step 203: Track the sight focus.

[0008] Furthermore, after step 201, the method further includes: Step 201a: determine whether the brightness of the facial image meets the preset brightness value. If yes, proceed to step 202; otherwise, correct the facial image.

[0009] Specifically, step 201a includes: Step A1: obtaining the original RGB frame of the face image, converting the original image frame into YCbCr space, and extracting the luminance component; Step A2: Calculate a histogram of the brightness component and obtain a first brightness value and a second brightness value in the histogram; Step A3: adjusting the brightness component according to the first brightness value and the second brightness value to obtain a corrected brightness; Step A4: Merge the corrected brightness with the Cb and Cr components of the original RGB frame to form a brightness-compensated YCbCr image, and convert it back to the RGB space to obtain a brightness-compensated RGB image.

[0010] Specifically, the said A3 includes: If the first brightness value is less than or equal to a first preset threshold or the second brightness value is greater than or equal to a second preset threshold, the brightness component is corrected according to a first preset algorithm; otherwise, the brightness component is corrected according to a second preset algorithm.

[0011] Specifically, the first preset algorithm is:

[0012] Wherein, Y(x,y) is the brightness component before correction, G(x,y) is the brightness component after correction, Y1 is the first brightness value, and Y2 is the second brightness value; The second preset algorithm is: Calculate the brightness mean m0 of the pre-stored template image and the brightness mean m of the current image c , if |m c If -m0|>5, adjust the brightness component Y(x,y) according to the following formula: .

[0013] Specifically, step 203 includes: Step B1, performing three-level downsampling on the original RGB frame to generate a first-level downsampled image, a second-level downsampled image, and a third-level downsampled image; Step B2: converting the target area of ​​the brightness-compensated RGB image into the HSV space, generating a chromaticity histogram, normalizing and sequentially arranging the chromaticity distribution probabilities to generate a chromaticity probability distribution set, and extracting a target allocation element from the set, wherein the target allocation element is the first k elements in the chromaticity probability distribution set, and the sum of the cumulative probability distributions of the first k elements is greater than a preset threshold; Step B3: determining the allocation elements corresponding to each downsampled image during back projection according to a first preset rule; Step B4: back-projecting the first group of allocated elements onto the three-level down-sampled image; Step B5: Detect non-zero connected areas in the current downsampled image and determine whether the candidate target center is detected. If so, proceed to step B6; otherwise, project the next set of assigned elements onto the corresponding downsampled image and repeat this step until the last set of assigned elements is detected. Step B6: upsampling the candidate target center to generate the target center; Step B7: Using the target center as the initial search window, calculating the centroid of the probability distribution within the window, iteratively moving the window until the centroid coincides with the center of gravity, and outputting a tracking result, the tracking result including the window position and bounding box size; Step B8: Use the tracking result of the current frame as the initial search window for the next frame.

[0014] Specifically, the first preset rule is: The first group of allocated elements corresponding to the three-level downsampled image is p1 to p k3 , where k3=⌈k / 3⌉, ⌈⌉ means rounding up; The second set of allocated elements corresponding to the secondary downsampled image is p k3+1 to p k3+k2 , where k2=⌊(k-k3) / 2⌋, where ⌊⌋ means round down; The third group of assigned elements corresponding to the first-level downsampled image is p k3+k1+1 to p k , where k1=k-k3-k2.

[0015] Specifically, the step 3 includes: Step 301: calibrate and match the sight focus with the pixel coordinates of the scene image; Step 302: Determine the pixel area to be enlarged as the target image area based on the corner points obtained in real time; Step 303: Enlarge the target image area by a preset multiple and display it in a second view.

[0016] The beneficial effects of the present invention are as follows: the present invention obtains the driver's line of sight focus, and when the line of sight focus falls within a preset range, the panoramic image of the area corresponding to the line of sight focus is magnified by a preset multiple and displayed, and obstacles can be marked and synchronously updated to other display devices, thereby facilitating the driver to obtain environmental information around the vehicle and improving driving safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is a flow chart of the panoramic system view control method of the present invention. DETAILED DESCRIPTION

[0018] The following describes the embodiments of the present invention in detail with reference to the accompanying drawings, which are for reference and illustration only and do not limit the scope of patent protection of the present invention.

[0019] In the processes described in the specification, claims, or drawings of the present invention, if the steps are numbered (e.g., steps 10, 20, etc.), these numbers are used solely to distinguish the steps and do not imply any order of execution. It should be noted that terms such as "first" and "second" are used solely to distinguish the objects being described and do not imply a sequential order or indicate different types of steps.

[0020] like Figure 1 As shown, this embodiment provides a panoramic system view control method, including: Step 1: Initialize the system and display a first view on a first target display device.

[0021] Step 2: Obtain the driver's visual focus.

[0022] Step 3: Determine whether the sight focus is within a preset range, and if so, display a second view corresponding to the sight focus on the first target display device.

[0023] In another embodiment of the present invention, after step 3, the method further includes: Step 4: Detect whether there is a target obstacle in the second view, and if so, mark the target obstacle.

[0024] In another embodiment of the present invention, after step 4, the method further includes: Step 5: Synchronously update the target obstacle information to the second target display device.

[0025] In a specific implementation, the first target display device is a central control display screen, and the second target display device is a HUD.

[0026] In another embodiment of the present invention, after step 5, the method further includes: Step 6: Determine whether the sight focus deviates from the current driving direction for more than a preset time, and if so, issue an alarm in a preset manner.

[0027] During specific implementation, the preset time and preset method can be set according to actual conditions. In this embodiment, the preset time is 3 to 5 seconds, and the preset method includes voice prompts and / or seat vibrations.

[0028] In another embodiment of the present invention, step 1 comprises: Step 101: Acquire a target video, and stitch the target video into a panoramic picture using a first preset algorithm.

[0029] Step 102: Display the panoramic image as a first view.

[0030] In a specific implementation, the first preset algorithm includes: a dynamic distortion compensation algorithm and an image stitching algorithm.

[0031] In another embodiment of the present invention, step 2 comprises: Step 201: Acquire the driver's facial image in real time.

[0032] Step 202: Recognize the facial image and obtain the driver's visual focus through a second preset algorithm.

[0033] Step 203: Track the sight focus.

[0034] In specific implementation, the second preset algorithm includes: a pupil center positioning algorithm and a head posture estimation algorithm.

[0035] In another embodiment of the present invention, after step 201, the method further includes: Step 201a: determine whether the brightness of the facial image meets the preset brightness value. If yes, proceed to step 202; otherwise, correct the facial image.

[0036] In another embodiment of the present invention, step 201a includes: Step A1: Obtain the original RGB frame of the face image, convert the original image frame into YCbCr space, and extract the brightness component Y(x,y).

[0037] Step A2: Calculate the histogram of the brightness component Y(x, y), and obtain the first brightness value Y1 and the second brightness value Y2 in the histogram.

[0038] The brightness histogram of an image is a bar graph. The horizontal axis represents the brightness level, from 0 (dark tones) on the left to 255 (light tones) on the right, dividing the brightness level of the photo into 256 levels; the vertical axis represents the number of pixels at each brightness level. The higher the peak value, the more pixels there are for that brightness value.

[0039] In this embodiment, the first brightness value Y1 is the brightness value corresponding to a pixel ratio of 5% when the brightness levels are counted from low to high, and the second brightness value Y2 is the brightness value corresponding to a pixel ratio of 95% when the brightness levels are counted from low to high.

[0040] For example, if the number of pixels with grayscales between 0 and 39 in the brightness histogram accounts for 4.8%, and the number of pixels with grayscales between 0 and 40 accounts for 5.2%, then Y1 = 40; if the number of pixels with grayscales between 0 and 199 in the brightness histogram accounts for 94.5%, and the number of pixels with grayscales between 0 and 200 accounts for 95.2%, then Y2 = 200.

[0041] Step A3: Adjust the brightness component Y(x, y) according to the first brightness value Y1 and the second brightness value Y2 to obtain a corrected brightness G(x, y).

[0042] In this embodiment, the inclusion A3 includes: If the first brightness value Y1 ≤ the first preset threshold T1 or the second brightness value Y2 ≥ the second preset threshold T2, the brightness component Y(x,y) is corrected according to the first preset algorithm; otherwise, the brightness component Y(x,y) is corrected according to the second preset algorithm.

[0043] In this embodiment, the first preset algorithm is:

[0044] Wherein, Y(x, y) is the brightness component before correction, G(x, y) is the brightness component after correction, Y1 is the first brightness value, and Y2 is the second brightness value.

[0045] In specific implementation, the first preset threshold T1 is much smaller than the second preset threshold T2. T1 and T2 can be calibrated according to experimental results. In this embodiment, the first preset threshold T1 is 40, and the second preset threshold T2 is 210.

[0046] In this embodiment, the second preset algorithm is: Calculate the brightness mean m0 of the pre-stored template image and the brightness mean m of the current image c , if |m c If -m0|>5, adjust the brightness component Y(x,y) according to the following formula: .

[0047] Step A4: Merge the corrected brightness G(x, y) with the Cb and Cr components of the original RGB frame into a brightness-compensated YCbCr image, and convert it back to the RGB space to obtain a brightness-compensated RGB image.

[0048] In another embodiment of the present invention, step 203 includes: Step B1, perform three-level downsampling on the original RGB frame to generate a first-level downsampling image img1, a second-level downsampling image img2, and a third-level downsampling image img3.

[0049] Among them, the first-level down-sampled image I ds1 The resolution is 1 / 2 of the original RGB frame image, and the secondary downsampled image I ds2 The resolution is 1 / 4 of the original RGB frame image, and the three-level down-sampled image I ds3 The resolution is 1 / 8 of the original RGB frame image.

[0050] Step B2: convert the target area of ​​the brightness-compensated RGB image into the HSV space, generate a chromaticity histogram, normalize the chromaticity distribution probability and arrange it in order, generate a chromaticity probability distribution set P, and extract the target allocation element from the set P, where the target allocation element is the first k elements in the chromaticity probability distribution set P, and the sum of the cumulative probability distributions of the first k elements is greater than a preset threshold.

[0051] In this embodiment, the target area is a rectangular area including the driver's eyes.

[0052] In specific implementation, the continuous chromaticity value (0°-360°) is divided into 16 discrete intervals (bins), each bin corresponds to a 22.5° color band, and the distribution ratio of pixels in the target area in each color band is counted to obtain the target chromaticity histogram.

[0053] It is easy to understand that the chromaticity probability distribution set P={p1,p2,…,p 16}, where elements p1, p2, ..., p 16 Indicates the pixel distribution ratio in each chromaticity interval.

[0054] During specific implementation, the preset threshold can be set according to experimental results. In this embodiment, the preset threshold is 90%.

[0055] Obviously, the target allocation elements are p1, p2, ..., p k .

[0056] Step B3: determining the allocation elements corresponding to each downsampled image during back projection according to a first preset rule.

[0057] The first preset rule is: The first group of allocated elements corresponding to the three-level downsampled image img3 is p1 to p k3 , where k3=⌈k / 3⌉, ⌈⌉ means rounding up; The second group of allocated elements corresponding to the secondary downsampled image img2 is p k3+1 to p k3+k2 , where k2=⌊(k-k3) / 2⌋, where ⌊⌋ means round down; The third group of assigned elements corresponding to the first-level downsampled image img1 is p k3+k1+1 to p k , where k1=k-k3-k2.

[0058] Step B4: Back-project the first group of allocated elements to the three-level down-sampled image img3.

[0059] Step B5: Detect non-zero connected areas in the current downsampled image and determine whether the candidate target center is detected. If so, proceed to step B6; otherwise, project the next set of assigned elements into the corresponding downsampled image and repeat this step until the last set of assigned elements is detected.

[0060] The determination of whether the candidate target center is detected includes: determining whether the area of ​​any of the non-zero connected regions is greater than a preset face value threshold St; if so, taking the largest connected region as the candidate target center (x1, y1); otherwise, determining that the candidate target center is not detected.

[0061] If the center of the candidate target can be located in the three-level downsampling image img3, there is no need to continue to locate the center of the candidate target in the two-level downsampling image img2 and the one-level downsampling image img1. Conversely, if the center of the candidate target cannot be located in the three-level downsampling image img3, continue to detect whether the center of the candidate target can be located in the two-level downsampling image img2. If not, locate the center of the candidate target in the one-level downsampling image img1.

[0062] Step B6: upsample the candidate target center to generate the target center.

[0063] For example, if the candidate target center (x1, y1) is detected in the three-level downsampled image img3, after upsampling to restore the original image resolution, the coordinates of the target center are (8x1, 8y1).

[0064] Step B7: Use the target center as the initial search window, calculate the centroid of the probability distribution within the window, iteratively move the window until the centroid coincides with the center of gravity, and output the tracking result, which includes the window position (x, y) and the bounding box size (w, h).

[0065] Step B8: Use the tracking result of the current frame as the initial search window for the next frame.

[0066] In another embodiment of the present invention, step 3 includes: Step 301: Calibrate and match the sight focus with the pixel coordinates of the scene image.

[0067] Step 302: Determine the pixel area to be enlarged as the target image area based on the corner points obtained in real time.

[0068] Step 303: Enlarge the target image area by a preset multiple and display it in a second view.

[0069] During specific implementation, the preset magnification factor can be set according to actual effects.

[0070] The above disclosure is only a preferred embodiment of the present invention and cannot be used to limit the scope of protection of the present invention. Therefore, equivalent changes made according to the scope of the patent application of the present invention are still within the scope covered by the present invention.

Claims

1. A panoramic system view control method, characterized in that: include: Step 1: Initialize the system and display a first view on a first target display device; Step 2: Obtain the driver's sight focus; Step 3: Determine whether the sight focus is within a preset range, and if so, display a second view corresponding to the sight focus on the first target display device.

2. The panoramic system view control method according to claim 1, characterized in that: After step 3, the method further includes: Step 4: Detect whether there is a target obstacle in the second view, and if so, mark the target obstacle; Step 5: Synchronously updating the target obstacle information to the second target display device; Step 6: Determine whether the sight focus deviates from the current driving direction for more than a preset time, and if so, issue an alarm in a preset manner.

3. The panoramic system view control method according to claim 1, characterized in that: The step 2 includes: Step 201: Acquire the driver's facial image in real time; Step 202: Recognize the facial image to obtain the driver's sight focus; Step 203: Track the sight focus.

4. The panoramic system view control method according to claim 3, characterized in that: After step 201, the method further includes: Step 201a: determine whether the brightness of the facial image meets the preset brightness value. If yes, proceed to step 202; otherwise, correct the facial image.

5. The panoramic system view control method according to claim 4, characterized in that: The step 201a includes: Step A1: obtaining the original RGB frame of the face image, converting the original image frame into YCbCr space, and extracting the luminance component; Step A2: Calculate a histogram of the brightness component and obtain a first brightness value and a second brightness value in the histogram; Step A3: adjusting the brightness component according to the first brightness value and the second brightness value to obtain a corrected brightness; Step A4: Merge the corrected brightness with the Cb and Cr components of the original RGB frame to form a brightness-compensated YCbCr image, and convert it back to the RGB space to obtain a brightness-compensated RGB image.

6. The panoramic system view control method according to claim 5, characterized in that: Said inclusion A3 includes: If the first brightness value is less than or equal to a first preset threshold or the second brightness value is greater than or equal to a second preset threshold, the brightness component is corrected according to a first preset algorithm; otherwise, the brightness component is corrected according to a second preset algorithm.

7. The panoramic system view control method according to claim 6, characterized in that: The first preset algorithm is: Wherein, Y(x,y) is the brightness component before correction, G(x,y) is the brightness component after correction, Y1 is the first brightness value, and Y2 is the second brightness value; The second preset algorithm is: Calculate the brightness mean m0 of the pre-stored template image and the brightness mean m of the current image c , if |m c If -m0|>5, adjust the brightness component Y(x,y) according to the following formula: 。 8. The panoramic system view control method according to claim 5, characterized in that: The step 203 includes: Step B1, performing three-level downsampling on the original RGB frame to generate a first-level downsampled image, a second-level downsampled image, and a third-level downsampled image; Step B2: converting the target area of ​​the brightness-compensated RGB image into the HSV space, generating a chromaticity histogram, normalizing and sequentially arranging the chromaticity distribution probabilities to generate a chromaticity probability distribution set, and extracting a target allocation element from the set, wherein the target allocation element is the first k elements in the chromaticity probability distribution set, and the sum of the cumulative probability distributions of the first k elements is greater than a preset threshold; Step B3: determining the allocation elements corresponding to each downsampled image during back projection according to a first preset rule; Step B4: back-projecting the first group of allocated elements onto the three-level down-sampled image; Step B5: Detect non-zero connected areas in the current downsampled image and determine whether the candidate target center is detected. If so, proceed to step B6; otherwise, project the next set of assigned elements onto the corresponding downsampled image and repeat this step until the last set of assigned elements is detected. Step B6: upsampling the candidate target center to generate the target center; Step B7: Using the target center as the initial search window, calculating the centroid of the probability distribution within the window, iteratively moving the window until the centroid coincides with the center of gravity, and outputting a tracking result, the tracking result including the window position and bounding box size; Step B8: Use the tracking result of the current frame as the initial search window for the next frame.

9. The panoramic system view control method according to claim 8, characterized in that: The first preset rule is: The first group of allocated elements corresponding to the three-level downsampled image is p1 to p k3 , where k3=⌈k / 3⌉, ⌈⌉ means rounding up; The second set of allocated elements corresponding to the secondary downsampled image is p k3+1 to p k3+k2 , where k2=⌊(k-k3) / 2⌋, where ⌊⌋ means round down; The third group of assigned elements corresponding to the first-level downsampled image is p k3+k1+1 to p k , where k1=k-k3-k2.

10. The panoramic system view control method according to claim 1, characterized in that: The step 3 includes: Step 301: calibrate and match the sight focus with the pixel coordinates of the scene image; Step 302: Determine the pixel area to be enlarged as the target image area based on the corner points obtained in real time; Step 303: Enlarge the target image area by a preset multiple and display it in a second view.

Images