A reversing image optimization and warning system

Abstract

The present invention belongs to the technical field of image processing, and discloses a reversing image optimization and warning system, comprising: an equipment control module, used for adaptively adjusting a vehicle-mounted camera of a target vehicle; a data acquisition module, used for acquiring images of a reversing process of the target vehicle based on the vehicle-mounted camera after adaptive adjustment; obtaining a corresponding reversing image sequence; an image processing module, used for performing image optimization processing on the obtained reversing image sequence to obtain a corresponding optimized image sequence; a risk supervision module, used for performing collision risk monitoring of the reversing process of the target vehicle based on the optimized image sequence, obtaining corresponding monitoring results, and taking corresponding risk warning measures based on the monitoring results; the present invention significantly improves the safety and convenience during the reversing process.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and more specifically, to a reversing image optimization and warning system. Background Art

[0002] With the development of the automobile industry and the improvement of people's living standards, more and more families own private cars. When reversing in a small parking space, due to limited vision, drivers often find it difficult to accurately judge the situation behind the vehicle, which makes collision accidents prone to occur.

[0003] Compared with the prior art, the reversing image systems currently on the market can provide a view of the rear of the vehicle, but have the following shortcomings: 1) The image quality is not high, especially in dim light or bad weather conditions, the image is blurry; 2) There is a lack of effective warning function and it is impossible to inform the driver of potential dangers in advance; In view of this, the present invention proposes a reversing image optimization and warning system to solve the above problems. Summary of the invention

[0004] In order to overcome the above-mentioned defects of the prior art and to achieve the above-mentioned purpose, the present invention provides the following technical solutions:

[0005] A reversing image optimization and warning system, comprising:

[0006] The equipment control module is used to adaptively adjust the onboard camera of the target vehicle;

[0007] The data acquisition module is used to acquire images of the target vehicle during the reversing process based on the adaptively adjusted vehicle-mounted camera, and obtain a corresponding reversing image sequence;

[0008] An image processing module is used to perform image optimization processing on the acquired reversing image sequence to obtain a corresponding optimized image sequence;

[0009] The risk supervision module is used to monitor the collision risk of the target vehicle during the reversing process based on the optimized image sequence, obtain corresponding monitoring results, and take corresponding risk warning measures based on the monitoring results.

[0010] Furthermore, the process of adaptively adjusting the onboard camera of the target vehicle includes:

[0011] When it is identified that the parking gear position of the target vehicle is the reverse gear position, data collection is performed on the basic camera parameters of the vehicle-mounted camera in the target vehicle and the environmental parameters thereof to obtain corresponding basic camera data and environmental data;

[0012] Inputting the collected basic camera data and its environmental data into a pre-built strategy adjustment model to obtain a corresponding parameter adjustment strategy, wherein the parameter adjustment strategy includes a first adjustment strategy and a second adjustment strategy;

[0013] Based on the first adjustment strategy and the second adjustment strategy, a corresponding optimal adjustment strategy is obtained and fed back to the vehicle management center of the target vehicle; after the vehicle management center receives the corresponding optimal adjustment strategy, parameters of the corresponding vehicle camera are corrected.

[0014] Furthermore, the construction process of the strategy adjustment model includes:

[0015] Constructing a strategy database, wherein the strategy database is composed of several groups of historical basic camera parameters and historical environmental data and their corresponding historical adjustment strategies;

[0016] Construct corresponding training data based on the strategy database, use historical basic camera parameters and historical environmental data in the training data as input vectors, and adjust the strategy as output vectors;

[0017] Define the backbone network of the strategy adjustment model as an improved convolutional neural network and introduce a reinforcement learning mechanism;

[0018] The iterative training formula of the reinforcement learning mechanism is:

[0019] Q yq (t) = (1-α) × Q yq (t)+α[R t+1 +γ×Q yq (t+1)];

[0020] In the formula, Q yq (t) = q(S t ,C t ,w t ), indicating that in the input vector S t ; Execute adjustment action C t The expected gain when t represents the reinforcement learning parameters at the current moment; α represents the learning efficiency, γ represents the discount efficiency; R t+1 It means to perform the adjustment action C under the state corresponding to the input vector t The instant bonus points brought by R t+1 =f(S t+1 );wherein, f() represents a linear function;

[0021] Define the loss function of the policy adjustment model In the formula, Q sj (t) indicates that the input vector is S t When the adjustment action C is executed tThe actual gain obtained when E sj [] expresses the mathematical expectation operation;

[0022] An optimizer is defined, and the optimizer performs model training on the corresponding policy adjustment model based on the constructed training samples until the corresponding loss function region converges; and the model parameters of the policy adjustment model at the last training of the optimizer are saved, thereby completing the training process of the policy adjustment model.

[0023] Furthermore, based on the adaptively adjusted on-board camera, the image of the target vehicle during the reversing process is collected; the process of obtaining the corresponding reversing image sequence includes:

[0024] When the on-board camera completes parameter correction, the on-board camera after parameter correction performs video capture of the target vehicle's reversing process to obtain a corresponding video stream; the captured video stream is read and divided into frame rates to obtain a number of single-frame reversing images, and the single-frame reversing images are sorted based on the capture time corresponding to the corresponding single-frame reversing images to obtain a corresponding reversing image sequence.

[0025] Furthermore, the obtained reversing image sequence is subjected to image optimization processing, and the process of obtaining the corresponding optimized image sequence includes:

[0026] Obtain a single-frame reversing image in the reversing image sequence, convert it into the HSV color space, and perform filtering processing on the channel images corresponding to each color channel in the corresponding HSV space to obtain the corresponding optimized channel images;

[0027] Performing image inversion on the obtained optimized channel image to obtain a corresponding inverted image;

[0028] Then, the corresponding inverted image is smoothed based on the guided filtering algorithm to obtain the corresponding initial image;

[0029] Calculate pixel deviations between the corresponding inverted image and the pixel points at corresponding positions in the detail image to obtain corresponding pixel deviations; and construct corresponding deviation images based on the pixel deviations;

[0030] Constructing a corresponding pixel correction factor based on the obtained initial image, and performing image enhancement processing on the corresponding optimized channel image based on the pixel correction factor to obtain a corresponding enhanced inverted image;

[0031] The enhanced inverted image obtained is fused with the corresponding pixel deviation image to obtain the corresponding optimized inverted image; the image fusion formula is: TX yv =μ×TX pc +TX zv ; In the formula, TX yv Indicates optimized inverted image; TXzv Indicates enhanced inverted image; TX pc represents the pixel deviation image, μ represents the image weight of the pixel deviation image;

[0032] Then, the obtained optimized channel image is converted from the HSV color space to the RGB color space to obtain the corresponding optimized reversing image;

[0033] The acquisition process of optimizing the reversing image is repeated until the optimized reversing images corresponding to all the reversing images in the corresponding reversing image sequence are acquired, and the corresponding optimized image sequence is obtained based on the optimized reversing images.

[0034] Furthermore, the process of obtaining the pixel correction factor includes:

[0035] Construct a blank window, arbitrarily select a pixel point in the inverted image as the center point of the blank window, and based on it, perform a region interception on the corresponding inverted image to obtain the corresponding inverted region, and obtain the minimum pixel value in the corresponding inverted region;

[0036] Repeat the above minimum pixel value acquisition process to obtain the minimum pixel value corresponding to several inversion areas; and record it as LD min (y); y = 1, 2, ..., Y, Y is a natural number, Y represents the total number of inversion regions; LD min (y) represents the minimum pixel value in the yth inversion area;

[0037] Then, based on the obtained minimum pixel value, the pixel correction factor is obtained. Where, ld fv Indicates the pixel correction factor corresponding to the inverted image; A indicates the camera parameter at the corresponding moment; min() indicates taking the minimum value; w1 indicates the pixel adjustment factor;

[0038] The mathematical calculation formula for the image enhancement process is: In the formula, TX zv (i) represents the pixel value at the pixel point corresponding to the pixel point i in the enhanced inverted image; t1 is a fixed constant; max() represents the maximum value.

[0039] Furthermore, the process of filtering the channel images corresponding to each color channel in the corresponding HSV space to obtain the corresponding optimized channel images includes:

[0040] Performing bilateral filtering on the channel image to obtain a corresponding first filtered image;

[0041] At the same time, the channel image is segmented to obtain P0 non-overlapping image blocks; two image blocks are read arbitrarily and recorded as p1 and p2, and the similarity distance between the corresponding image blocks is obtained. In the formula, TX p1 and TX p2 Respectively represent the pixel values ​​of the central pixels in the read image blocks p1 and p2; M and N represent the length and width of the channel image respectively; ‖.‖ represents the Euclidean distance operation;

[0042] Set a distance threshold, and compare the obtained similarity distance with the distance threshold; if the similarity distance is less than or equal to the distance threshold, no other operations are performed; if the similarity distance is greater than the distance threshold, it indicates that image blocks p1 and p2 are similar image blocks, and they are divided into the same block group;

[0043] Based on the above block grouping division process, all image blocks are divided to obtain a plurality of image block groups;

[0044] Get the image blocks in the image group and perform image superposition on them to obtain a three-dimensional image matrix JZ c Among them, JZ c represents the three-dimensional image matrix corresponding to the c-th image group; c = 1, 2, ..., B; B is a natural number, and B represents the total number of image groups;

[0045] Perform matrix transformation on the obtained three-dimensional image matrix to obtain the image estimation matrix of the corresponding image group; the formula for matrix transformation is: In the formula, and Γ 3D () represent the three-dimensional inverse transform and three-dimensional transform respectively; ξ represents the hard threshold filtering operation; Represents the three-dimensional image matrix JZ c The corresponding image estimation matrix; obtaining the corresponding estimated area image based on the image estimation matrix;

[0046] Performing weighted summation on the image block and the corresponding estimated region image to obtain a corresponding optimized image block, and replacing the original image block in the corresponding channel image;

[0047] Based on the acquisition process of the corresponding optimized image blocks, all image blocks in the channel image are traversed, and the optimized image blocks are used to replace the original image blocks. After the replacement is completed, the channel image after the replacement is recorded as the second filtered image;

[0048] The first filtered image and the second filtered image are weightedly fused to obtain the corresponding optimized channel image; wherein the weighted fusion formula is: TX yh =ω×TX lb1 +(1-ω)TXlb2 ; In the formula, TX yh Indicates optimized channel image; TX lb1 and TX lb2 represent the first filtered image and the second filtered image respectively; ω represents the fusion weight corresponding to the first filtered image.

[0049] Furthermore, the formula for bilateral filtering is: In the formula, TX dl (a, b) represents the pixel value at the pixel point (a, b) in the first filtered image; Z represents the filter size selected in the corresponding bilateral filtering process; TX td (m, n) represents the pixel value of the pixel at coordinate (m, n) adjacent to the pixel (a, b) in the channel image; W(a, b, m, n) represents the weight coefficient of the filter; where W(a, b, m, n) = U(a, b, m, n) × R(a, b, m, n); where U(a, b, m, n) and R(a, b, m, n) represent the spatial kernel function and the color kernel function, respectively; where; Where F(a, b) represents the pixel value at the pixel point (a, b) in the channel image; ε R and ε U Represent the spatial proximity factor and color similarity factor respectively.

[0050] Furthermore, the process of performing collision risk monitoring on the reversing process of the target vehicle based on the optimized image sequence and obtaining corresponding monitoring results includes:

[0051] Obtain the median value of the pixel values ​​corresponding to the last row of pixel points in the corresponding optimized reversing image, and record it as the road surface reference pixel value. At the same time, mark the pixel point corresponding to the corresponding median value as the road surface reference point;

[0052] The Euclidean distances between other pixel points in the optimized reversing image and the corresponding road reference points are respectively obtained, and based on the Euclidean distances, pixel point estimation is performed on other pixel points in the corresponding optimized reversing image; and the corresponding estimated pixel values ​​are obtained;

[0053] The obtained estimated pixel value is calculated to obtain a corresponding pixel deviation by comparing the pixel value of the corresponding pixel point in the optimized reversing image;

[0054] A deviation threshold is set, and the obtained pixel deviation is compared with the corresponding deviation threshold. If the pixel deviation is less than or equal to the deviation threshold, the corresponding pixel point is marked as a road surface similarity point; if the pixel deviation is greater than the deviation threshold, the corresponding pixel point is marked as an object suspected point;

[0055] Based on the suspected points of the object and in combination with the contour extraction technology, the corresponding suspected contour of the object is obtained; and the circumscribed rectangle of the corresponding suspected contour of the object is marked as the suspected area of ​​the object;

[0056] Obtain pixel coordinates of diagonal pixels in the suspected object area, and based on them, obtain the pixel height of the corresponding suspected object area; and compare it with the preset height threshold; if the pixel height is less than the pixel threshold, no other operations are performed; if the pixel height is not less than the height threshold, the corresponding suspected object area is marked as an obstacle area;

[0057] Obtain pixel coordinates in the corresponding obstacle area and perform coordinate transformation on them to obtain obstacle coordinate information of the obstacle object corresponding to the corresponding obstacle area in the world coordinate system;

[0058] At the same time, the three-dimensional coordinates of the camera on the corresponding vehicle at the time when the corresponding optimized reversing image is located are obtained; and the first safety distance is obtained based on the three-dimensional coordinates of the camera and the obstacle coordinate information;

[0059] At the same time, ultrasonic ranging is performed based on the target vehicle's onboard radar and combined with the obstacle coordinate information to obtain the corresponding second safety distance;

[0060] Then, the vehicle speed of the target vehicle is read, and the safety distance of the next frame of the optimized reversing image is predicted in combination with the first safety distance and the second safety distance, and the safety distance is marked as the target safety distance, and the corresponding target safety distance is marked as a distance in the read optimized reversing image;

[0061] After the distance marking is completed, the optimized image sequence for distance marking is converted into a video format, and displayed to the vehicle user based on the on-board display of the target vehicle;

[0062] At the same time; set the collision range, and take corresponding early warning measures based on the collision safety range of the target safety distance.

[0063] Furthermore, the estimated pixel value Where D g0 represents the Euclidean distance between the g-th pixel point in the optimized reversing image and the corresponding road reference point, where g = 0, 1, 2, ..., G, G represents the total number of pixels in the optimized reversing image; G is a natural number, and when g = 0, g represents the road reference point; JD0 represents the road reference pixel value; sp(g) and sz(g) represent the horizontal pixel distance and vertical pixel distance between the pixel point g and the road reference point, respectively; ρ2 and ρ1 represent the distance weights of the vertical pixel distance and the horizontal pixel distance, respectively.

[0064] The technical effects and advantages of the reversing image optimization and warning system of the present invention are as follows:

[0065] 1. Adaptively adjust the vehicle-mounted camera through the equipment control module to ensure that the camera can capture clear and accurate reversing images under different environmental conditions, providing a reliable visual reference for the driver;

[0066] At the same time, the reversing image sequence is subjected to image filtering and enhancement processing to effectively remove noise and interference in the image, improve image clarity and contrast, and enable the driver to observe the surrounding environment more clearly; the detailed information in the image is retained, and the optimized image is combined with the detailed image through image fusion technology to further enhance the visual effect of the image.

[0067] 2. Obtaining the obstacle area in the reversing image in the current frame and combining it with the position of the on-board camera to predict the target safety distance is helpful to increase the reaction time of the vehicle user and generate early warning information in time to remind the driver to avoid it and effectively prevent the occurrence of reversing accidents;

[0068] 3. By reading the optimized image sequence, identifying feature points and calculating the actual distance between them and the on-board camera, and using the distance information obtained by the on-board millimeter radar to determine whether there is a reversing risk. This dual verification method improves the accuracy of collision risk judgment. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] Figure 1 A schematic diagram of a reversing image optimization and warning system of the present invention;

[0070] Figure 2 It is a schematic diagram of a reversing image optimization and warning method of the present invention. DETAILED DESCRIPTION

[0071] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0072] Example 1

[0073] See also Figure 1 As shown, the reversing image optimization and warning system described in this embodiment includes:

[0074] The equipment control module is used to adaptively adjust the onboard camera of the target vehicle;

[0075] The data acquisition module is used to acquire images of the target vehicle during the reversing process based on the adaptively adjusted vehicle-mounted camera, and obtain a corresponding reversing image sequence;

[0076] An image processing module is used to perform image optimization processing on the acquired reversing image sequence to obtain a corresponding optimized image sequence;

[0077] A risk monitoring module is used to monitor the collision risk of the target vehicle during the reversing process based on the optimized image sequence, obtain corresponding monitoring results, and take corresponding risk warning measures based on the monitoring results;

[0078] The modules are connected via wired and / or wireless means to achieve data transmission between the modules.

[0079] It should be further explained that, in the specific implementation process, the process of adaptively adjusting the onboard camera of the target vehicle includes:

[0080] When the device control module recognizes that the parking gear of the target vehicle is the reverse gear, the device control module collects data on the basic camera parameters of the vehicle-mounted camera in the target vehicle and the environmental parameters thereof, and obtains corresponding basic camera data and environmental data; wherein the basic camera data includes camera temperature, camera humidity, camera exposure, etc.; the environmental data includes environmental pixels, environmental humidity, environmental temperature, etc.;

[0081] Inputting the collected basic camera data and its environmental data into a pre-built strategy adjustment model to obtain a corresponding parameter adjustment strategy, wherein the parameter adjustment strategy includes a first adjustment strategy and a second adjustment strategy;

[0082] Respectively obtaining the first camera parameter and the second camera parameter corresponding to the first adjustment strategy and the second adjustment strategy, and performing deviation calculation on them to obtain corresponding parameter deviation intervals; wherein the first camera parameter and the second camera parameter respectively represent the optimal and suboptimal parameter adjustment values ​​of the vehicle-mounted camera predicted by the strategy adjustment model based on the currently input environmental data and basic camera data;

[0083] Furthermore, based on the parameter deviation interval, the first camera parameter in the corresponding first adjustment strategy is subjected to deviation adjustment, and the expected gain corresponding to the first adjustment strategy after the deviation adjustment is obtained, and the expected gain is compared with the original expected gain corresponding to the first adjustment strategy without parameter adjustment; if the expected gain after the corresponding parameter adjustment is not higher than the original expected gain, the original first adjustment strategy is used as the optimal adjustment strategy and fed back to the vehicle management center of the target vehicle; if the expected gain after the corresponding parameter adjustment is higher than the original expected gain, the first adjustment strategy after the corresponding parameter adjustment is used as the optimal adjustment strategy and fed back to the vehicle management center of the target vehicle; when the vehicle management center receives the corresponding optimal adjustment strategy, the parameters of the corresponding vehicle-mounted camera are corrected; wherein, the optimal adjustment strategy is updated in real time based on the collected basic camera data and environmental data, and the vehicle-mounted camera is synchronously corrected based on the optimal adjustment strategy updated in real time;

[0084] It should be further explained that, in the specific implementation process, the construction process of the strategy adjustment model includes:

[0085] Constructing a strategy database, the strategy database is composed of several groups of historical basic camera parameters and historical environmental data and their corresponding historical adjustment strategies, the historical adjustment strategy refers to the corresponding adjustment action and the basic camera parameters of the vehicle camera after adjustment during the operation of the same type of vehicle camera based on the basic camera parameters and environmental data; the historical adjustment strategy acquisition method can also be the adjustment strategy set for the corresponding vehicle camera under different environmental data tested by the corresponding staff during the factory test of the corresponding vehicle camera;

[0086] Construct corresponding training data based on the strategy database, use historical basic camera parameters and historical environmental data in the training data as input vectors, and adjust the strategy as output vectors;

[0087] The backbone network of the strategy adjustment model is defined as an improved convolutional neural network, and a reinforcement learning mechanism is introduced to improve the efficiency of the improved convolutional neural network in learning the best decision-making strategy, and also help it have complex decision-making capabilities;

[0088] The iterative training formula of the reinforcement learning mechanism is:

[0089] Q yq (t) = (1-α) × Q yq (t)+α[R t+1 +γ×Q yq (t+1)];

[0090] In the formula, Q yq (t) = q(S t , Ct , w t ), indicating that in the input vector S t ; Execute adjustment action C t The expected gain when t represents the reinforcement learning parameters at the current moment; α represents the learning efficiency, γ represents the discount efficiency; R t+1 It means to perform the adjustment action C under the state corresponding to the input vector t The instant bonus points brought by R t+1 =f(S t+1 ), where f() represents a function used to evaluate the input vector S t+1 The quality of the state; for example, if the state is temperature, then f() can be a simple linear function that returns a positive value if the temperature is within the ideal range and a negative value otherwise;

[0091] Define the loss function of the policy adjustment model In the formula, Q sj (t) indicates that the input vector is S t When the adjustment action C is executed t The actual gain obtained when E sj [] expresses the mathematical expectation operation;

[0092] An optimizer is defined, and the optimizer performs model training on the corresponding policy adjustment model based on the constructed training samples until the corresponding loss function region converges; and the model parameters of the policy adjustment model at the last training of the optimizer are saved, thereby completing the training process of the policy adjustment model.

[0093] It should be further explained that, in the specific implementation process, the image of the target vehicle's reversing process is collected based on the adaptively adjusted vehicle-mounted camera; the process of obtaining the corresponding reversing image sequence includes:

[0094] When the on-board camera completes parameter correction, the data acquisition module performs video acquisition on the target vehicle's reversing process based on the on-board camera after parameter correction to obtain a corresponding video stream; then, the data acquisition module reads the acquired video stream and divides it according to the frame rate to obtain a number of single-frame reversing images, and sorts them based on the acquisition time corresponding to the corresponding single-frame reversing images to obtain a corresponding reversing image sequence.

[0095] It should be further explained that, in the specific implementation process, the image optimization processing is performed on the obtained reversing image sequence to obtain the corresponding optimized image sequence, which includes:

[0096] Obtain a single-frame reversing image in the reversing image sequence, convert it into the HSV color space, and perform filtering processing on the channel images corresponding to each color channel in the corresponding HSV space to obtain the corresponding optimized channel images;

[0097] The obtained optimized channel image is subjected to image inversion to obtain a corresponding inverted image; the mathematical formula for the corresponding image inversion is: TX fv (i) = 255-TX v (i); TX v (i) represents the pixel value corresponding to pixel i in the optimized channel image; TX fv (i) represents the pixel value of the pixel point at the position corresponding to the pixel point i in the inverted image; wherein i=1, 2, ..., P; P is a natural number, and P represents the total number of pixels in the corresponding optimized channel image;

[0098] Then, the corresponding inverted image is smoothed based on the guided filtering algorithm to obtain the corresponding initial image; wherein the smoothing refers to extracting the image details in the inverted image based on the guided filtering algorithm; this is the prior art and will not be elaborated in detail in the present invention;

[0099] Calculate pixel deviations of the corresponding inverted image and pixel points at corresponding positions in the detail image to obtain corresponding pixel deviations; and construct corresponding deviation images based on the pixel deviations; the pixel deviation image refers to the image detail portion removed when the corresponding inverted image is smoothed;

[0100] Constructing a corresponding pixel correction factor based on the obtained initial image, and performing image enhancement processing on the corresponding optimized channel image based on the pixel correction factor to obtain a corresponding enhanced inverted image;

[0101] The process of obtaining the pixel correction factor includes:

[0102] Construct a blank window, arbitrarily select a pixel point in the inverted image as the center point of the blank window, and based on it, perform a region interception on the corresponding inverted image to obtain the corresponding inverted region, and obtain the minimum pixel value in the corresponding inverted region;

[0103] Repeat the above minimum pixel value acquisition process to obtain the minimum pixel value corresponding to several inversion areas; and record it as LD min (y); y = 1, 2, ..., Y, Y is a natural number, indicating the total number of Y-th inversion regions; LD min (y) represents the minimum pixel value in the yth inversion area;

[0104] Then, based on the obtained minimum pixel value, the corresponding pixel correction factor is obtained. Where, ld fv Indicates the pixel correction factor corresponding to the inverted image; A indicates the camera parameter corresponding to the current moment, and the camera parameter is one of the saturation, exposure and hue of the corresponding vehicle-mounted camera when collecting the corresponding single reversing image; min() indicates taking the minimum value; w1 indicates the pixel adjustment factor; it depends on the actual needs;

[0105] The mathematical calculation formula for the image enhancement process is: In the formula, TX zv (i) represents the pixel value at the pixel point corresponding to the pixel point i in the enhanced inverted image; t1 is a fixed constant used to avoid the denominator being zero; max() represents the maximum value;

[0106] Then, the obtained enhanced inverted image is fused with the corresponding pixel deviation image to obtain the corresponding optimized inverted image; the image fusion formula is: TX yv =μ×TX pc +TX zv ; In the formula, TX yv Indicates optimized inverted image; TX zv Indicates enhanced inverted image; TX pc represents the pixel deviation image, μ represents the image weight of the pixel deviation image;

[0107] Then, the obtained optimized channel image is converted from the HSV color space to the RGB color space to obtain the corresponding optimized reversing image.

[0108] It should be further explained that, in the specific implementation process, the channel images corresponding to each color channel in the corresponding HSV space are filtered respectively, and the process of obtaining the corresponding optimized channel image includes:

[0109] Taking a certain color channel as an example, reading a channel image under the corresponding color channel, performing bilateral filtering on the channel image, and obtaining a corresponding first filtered image;

[0110] The corresponding bilateral filtering formula is: In the formula, TX dl (a, b) represents the pixel value at the pixel point (a, b) in the first filtered image; Z represents the filter size selected in the corresponding bilateral filtering process; TX td(m, n) represents the pixel value of the pixel at coordinate (m, n) adjacent to the pixel (a, b) in the channel image; W(a, b, m, n) represents the weight coefficient of the filter; where W(a, b, m, n) = U(a, b, m, n) × R(a, b, m, n); where U(a, b, m, n) and R(a, b, m, n) represent the spatial kernel function and the color kernel function, respectively; where; Where F(a,b) represents the pixel value at the pixel point (a,b) in the channel image; ε R and ε U Respectively represent the spatial proximity factor and color similarity factor; they are key factors affecting the filter effect. When the values ​​of the two influencing factors are too large, bilateral filtering will be converted to mean filtering; conversely, when the values ​​are too small, the effect of smoothing the image cannot be achieved;

[0111] At the same time, the channel image is segmented to obtain P0 non-overlapping image blocks; two image blocks are read arbitrarily and recorded as p1 and p2, and the similarity distance between the corresponding image blocks is obtained. In the formula, TX p1 and TX p2 Respectively represent the pixel values ​​of the central pixels in the read image blocks p1 and p2; M and N represent the length and width of the channel image respectively; ‖.‖ represents the Euclidean distance operation;

[0112] Set a distance threshold, and compare the obtained similarity distance with the distance threshold; if the similarity distance is less than or equal to the distance threshold, no other operations are performed; if the similarity distance is greater than the distance threshold, it indicates that image blocks p1 and p2 are similar image blocks, and they are divided into the same block group;

[0113] Based on the above block grouping division process, all image blocks are divided to obtain a number of image block groups; wherein each image group includes at least two similar image blocks;

[0114] Get the image blocks in the image group and perform image superposition on them to obtain a three-dimensional image matrix JZ c Among them, JZ c represents the three-dimensional image matrix corresponding to the c-th image group; c = 1, 2, ..., B; B is a natural number used to represent the total number of image groups;

[0115] Perform matrix transformation on the obtained three-dimensional image matrix to obtain the image estimation matrix of the corresponding image group; the formula for matrix transformation is: In the formula, and Γ 3D () represent the three-dimensional inverse transform and three-dimensional transform respectively; ξ represents the hard threshold filtering operation; Represents the three-dimensional image matrix JZ c The corresponding image estimation matrix;

[0116] Furthermore, a corresponding estimated region image is obtained based on the image estimation matrix, wherein the estimated region image refers to an estimated image obtained after matrix transformation of image blocks in the corresponding image group; the estimated image is a matrix element in the corresponding image estimation matrix;

[0117] Then, the image block is weightedly summed with the corresponding estimated region image to obtain a corresponding optimized image block, and the original image block is replaced in the corresponding channel image;

[0118] Based on the acquisition process of the corresponding optimized image blocks, all image blocks in the channel image are traversed, and the optimized image blocks are used to replace the original image blocks. After the replacement is completed, the channel image after the replacement is recorded as the second filtered image;

[0119] The first filtered image and the second filtered image are weightedly fused to obtain the corresponding optimized channel image; wherein the weighted fusion formula is: TX yh =ω×TX lb1 +(1-ω)TX lb2 ; In the formula, TX yh Indicates optimized channel image; TX lb1 and TX lb2 denote the first filtered image and the second filtered image respectively; ω denotes the fusion weight corresponding to the first filtered image;

[0120] The acquisition process of optimizing the reversing image is repeated until the optimized reversing images corresponding to all the reversing images in the corresponding reversing image sequence are acquired, and the corresponding optimized image sequence is obtained based on the optimized reversing images.

[0121] It should be further explained that, in a specific implementation process, the collision risk monitoring of the target vehicle during the reversing process is performed based on the optimized image sequence, and the process of obtaining the corresponding monitoring result includes:

[0122] Randomly read a certain optimized reversing image in the optimized image sequence, obtain the median value of the pixel values ​​corresponding to the last row of pixels in the corresponding optimized reversing image, and record it as the road surface reference pixel value, and at the same time, mark the pixel point corresponding to the corresponding median value as the road surface reference point;

[0123] Obtain the Euclidean distances between other pixel points in the optimized reversing image and the corresponding road reference points respectively, and estimate the pixel points of other pixel points in the corresponding optimized reversing image based on the Euclidean distances; obtain the corresponding estimated pixel values Where D g0represents the Euclidean distance between the g-th pixel point in the optimized reversing image and the corresponding road reference point, where g = 0, 1, 2, ..., G, G represents the total number of pixels in the optimized reversing image; G is a natural number, and when g = 0, g represents the road reference point; JD0 represents the road reference pixel value; sp(g) and sz(g) represent the horizontal pixel distance and vertical pixel distance between the pixel point g and the road reference point, respectively; ρ2 and ρ1 represent the distance weights of the vertical pixel distance and the horizontal pixel distance, respectively;

[0124] The obtained estimated pixel value is calculated to obtain a corresponding pixel deviation by comparing the pixel value of the corresponding pixel point in the optimized reversing image;

[0125] A deviation threshold is set, and the obtained pixel deviation is compared with the corresponding deviation threshold. If the pixel deviation is less than or equal to the deviation threshold, the corresponding pixel point is marked as a road surface similarity point; if the pixel deviation is greater than the deviation threshold, the corresponding pixel point is marked as an object suspected point;

[0126] Based on the suspected points of the object and in combination with the contour extraction technology, the corresponding suspected contour of the object is obtained; and the circumscribed rectangle of the corresponding suspected contour of the object is marked as the suspected area of ​​the object;

[0127] Obtain pixel coordinates of diagonal pixel points in the suspected object area, and based on them, obtain the pixel height of the corresponding suspected object area; and compare it with a preset height threshold; if the pixel height is less than the pixel threshold, no other operation is performed; if the pixel height is not less than the height threshold, the corresponding suspected object area is marked as an obstacle area; wherein the height threshold is jointly determined by the ground clearance of the corresponding vehicle-mounted camera, the camera focal length, and the ground clearance of the vehicle chassis of the target vehicle;

[0128] Obtain pixel coordinates in the corresponding obstacle area, and perform coordinate transformation on them to obtain obstacle coordinate information of the obstacle object corresponding to the corresponding obstacle area in the world coordinate system; the obstacle coordinate information refers to the three-dimensional coordinates of the pixel coordinates in the corresponding obstacle area in the corresponding world coordinate system;

[0129] At the same time, the three-dimensional coordinates of the corresponding vehicle-mounted camera at the time when the corresponding optimized reversing image is located are obtained; and the physical obstacle distance between the corresponding vehicle-mounted camera and the corresponding obstacle area is obtained based on the three-dimensional coordinates of the camera and the obstacle coordinate information;

[0130] Obtain the minimum value of the physical obstacle distances between the vehicle-mounted camera and all obstacle areas, and mark it as the first safe distance;

[0131] At the same time, ultrasonic ranging is performed based on the target vehicle's onboard radar and combined with the obstacle coordinate information to obtain the corresponding second safety distance;

[0132] Then, the vehicle speed of the target vehicle is read, and the safety distance of the next frame of optimized reversing image is predicted in combination with the first safety distance and the second safety distance, and the safety distance is marked as the target safety distance, and the corresponding target safety distance is marked in the read optimized reversing image; (that is, the target safety distance marked in the corresponding optimized reversing image is actually the predicted safety distance corresponding to the next frame of optimized image);

[0133] After the distance marking is completed, the optimized image sequence for distance marking is converted into a video format, and displayed to the vehicle user based on the on-board display of the target vehicle;

[0134] At the same time; setting a collision range, the collision range includes a first collision range, a second collision range and a third collision range;

[0135] If the corresponding target safety distance is within the first collision range, no other operations are performed;

[0136] If the corresponding target safety distance is within the second collision range, when the vehicle-mounted display shows the corresponding optimized reversing image, the corresponding vehicle-mounted display frame is displayed in yellow, and a sound warning of a specific frequency is given;

[0137] If the corresponding target safety distance is within the third-dimensional collision range, when the vehicle-mounted display shows the corresponding optimized reversing image, the corresponding vehicle-mounted display frame is displayed in red and a continuous sound warning is given; among them, the smaller the target safety distance corresponding to the target vehicle, the darker the color display depth of the corresponding display frame, and the higher the frequency of the corresponding sound warning.

[0138] The present invention significantly improves the safety and reliability of the reversing process through technical means such as adaptive camera adjustment, image optimization processing, and intelligent decision support, and provides the driver with clearer and more accurate visual reference and intelligent assistance.

[0139] Example 2

[0140] See also Figure 2 As shown, the part not described in detail in this embodiment is described in Example 1, which provides a reversing image optimization and warning method, including:

[0141] Step 1: Adaptively adjust the onboard camera of the target vehicle;

[0142] Step 2: Based on the adaptively adjusted on-board camera, image acquisition is performed on the reversing process of the target vehicle to obtain a corresponding reversing image sequence;

[0143] Step 3: performing image optimization processing on the obtained reversing image sequence to obtain a corresponding optimized image sequence;

[0144] Step 4: Based on the optimized image sequence, the collision risk of the target vehicle during the reversing process is monitored to obtain corresponding monitoring results, and corresponding risk warning measures are taken based on the monitoring results.

[0145] Example 3

[0146] This embodiment discloses an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the operation mode of the reversing image optimization and warning system provided above is implemented.

[0147] Since the electronic device introduced in this embodiment is an electronic device used to implement a reversing image optimization and warning system in the embodiment of this application, based on the reversing image optimization and warning system introduced in the embodiment of this application, the technical personnel of this field can understand the specific implementation of the electronic device of this embodiment and its various variations, so how the electronic device implements the method in the embodiment of this application is not described in detail here. As long as the technical personnel of this field implement the electronic device used in the reversing image optimization and warning system in the embodiment of this application, it belongs to the scope of protection of this application.

[0148] The above formulas are all dimensionless and numerical calculations. The formula is a formula for the most recent real situation obtained by collecting a large amount of data and performing software simulation. The preset parameters and thresholds in the formula are set by technicians in this field according to actual conditions.

[0149] The above is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technical users in this technical field, some improvements and modifications without departing from the principle of the present invention should also be regarded as the protection scope of the present invention.

Claims

1. A reversing image optimization and warning system, characterized in that: include: The equipment control module is used to adaptively adjust the onboard camera of the target vehicle; The data acquisition module acquires images of the target vehicle during the reversing process based on the adaptively adjusted on-board camera, and obtains the corresponding reversing image sequence; An image processing module is used to perform image optimization processing on the acquired reversing image sequence to obtain a corresponding optimized image sequence; A risk monitoring module monitors the collision risk of the target vehicle during its reversing process based on the optimized image sequence, obtains corresponding monitoring results, and takes corresponding risk warning measures based on the monitoring results; The process of adaptively adjusting the target vehicle's onboard camera includes: When it is identified that the parking gear position of the target vehicle is the reverse gear position, data collection is performed on the basic camera parameters of the vehicle-mounted camera in the target vehicle and the environmental parameters thereof to obtain corresponding basic camera data and environmental data; Inputting the collected basic camera data and its environmental data into a pre-built strategy adjustment model to obtain a corresponding parameter adjustment strategy, wherein the parameter adjustment strategy includes a first adjustment strategy and a second adjustment strategy; Based on the first adjustment strategy and the second adjustment strategy, a corresponding optimal adjustment strategy is obtained, and the optimal adjustment strategy is fed back to the vehicle management center of the target vehicle; after the vehicle management center receives the optimal adjustment strategy, parameters of the corresponding vehicle camera are corrected; The image acquisition of the target vehicle during the reversing process is performed based on the adaptively adjusted on-board camera; the process of obtaining the corresponding reversing image sequence includes: When the on-board camera completes parameter correction, the on-board camera after parameter correction performs video capture of the target vehicle's reversing process to obtain a corresponding video stream; the captured video stream is read and frame rate divided to obtain a plurality of single-frame reversing images, and the single-frame reversing images are sorted based on the acquisition time corresponding to the respective single-frame reversing images to obtain a corresponding reversing image sequence; The process of performing image optimization processing on the obtained reversing image sequence to obtain the corresponding optimized image sequence includes: Obtain a single-frame reversing image in the reversing image sequence, convert it into the HSV color space, and perform filtering processing on the channel images corresponding to each color channel in the corresponding HSV space to obtain the corresponding optimized channel images; Performing image inversion on the obtained optimized channel image to obtain a corresponding inverted image; Then, the corresponding inverted image is smoothed based on the guided filtering algorithm to obtain the corresponding initial image; Calculate pixel deviations between the corresponding inverted image and the pixel points at corresponding positions in the detail image to obtain corresponding pixel deviations; and construct corresponding deviation images based on the pixel deviations; Constructing a corresponding pixel correction factor based on the obtained initial image, and performing image enhancement processing on the corresponding optimized channel image based on the pixel correction factor to obtain a corresponding enhanced inverted image; The enhanced inverted image obtained is fused with the corresponding pixel deviation image to obtain the corresponding optimized inverted image; the image fusion formula is: TX yv =μ×TX pc +TX zv ; In the formula, TX yv Indicates optimized inverted image; TX zv Indicates enhanced inverted image; TX pc represents the pixel deviation image, μ represents the image weight of the pixel deviation image; Then, the obtained optimized channel image is converted from the HSV color space to the RGB color space to obtain the corresponding optimized reversing image; The acquisition process of optimizing the reversing image is repeated until the optimized reversing images corresponding to all the reversing images in the corresponding reversing image sequence are acquired, and the corresponding optimized image sequence is obtained based on the optimized reversing images.

2. The reversing image optimization and warning system according to claim 1, characterized in that: The construction process of the strategy adjustment model includes: Constructing a strategy database, wherein the strategy database is composed of several groups of historical basic camera parameters and historical environmental data and their corresponding historical adjustment strategies; Construct corresponding training data based on the strategy database, use historical basic camera parameters and historical environmental data in the training data as input vectors, and adjust the strategy as output vectors; Define the backbone network of the strategy adjustment model as an improved convolutional neural network and introduce a reinforcement learning mechanism; The iterative training formula of the reinforcement learning mechanism is: Q yq (t)=(1-α)×Q yq (t)+α[R t+1 +γ×Q yq (t+1)]; In the formula, Q yq (t) = q(S t ,C t ,w t ), indicating that in the input vector S t Perform adjustment action C in state t The expected gain when t represents the reinforcement learning parameters at the current moment; α represents the learning efficiency, γ represents the discount efficiency; R t+1 It means to perform the adjustment action C under the state corresponding to the input vector t The instant bonus points brought by R t+1 =f(S t+1 );wherein, f() represents a linear function; Define the loss function of the policy adjustment model In the formula, Q sj (t) indicates that the input vector is S t When the adjustment action C is executed t The actual gain obtained when E sj [] expresses the mathematical expectation operation; An optimizer is defined, and the optimizer performs model training on the corresponding policy adjustment model based on the constructed training samples until the corresponding loss function region converges; and the model parameters of the policy adjustment model at the last training of the optimizer are saved, thereby completing the training process of the policy adjustment model.

3. The reversing image optimization and warning system according to claim 2, characterized in that: The process of obtaining the pixel correction factor includes: Construct a blank window, arbitrarily select a pixel point in the inverted image as the center point of the blank window, and based on it, perform a region interception on the corresponding inverted image to obtain the corresponding inverted region, and obtain the minimum pixel value in the corresponding inverted region; Repeat the above minimum pixel value acquisition process to obtain the minimum pixel value corresponding to several inversion areas; and record it as LD min (y); y = 1, 2, ..., Y, Y is a natural number, indicating the total number of Y-th inversion regions; LD min (y) represents the minimum pixel value in the yth inversion area; Then, based on the obtained minimum pixel value, the pixel correction factor is obtained. Where, ld fv Indicates the pixel correction factor corresponding to the inverted image; A indicates the camera parameter at the corresponding moment; min() indicates taking the minimum value; w1 indicates the pixel adjustment factor; The mathematical calculation formula for the image enhancement process is: In the formula, TX zv (i) represents the pixel value at the pixel point corresponding to the pixel point i in the enhanced inverted image; t1 is a fixed constant; max() represents the maximum value; TX fv (i) represents the pixel value of the pixel point corresponding to the pixel point i in the inverted image; wherein, i=1, 2, ..., P; P is a natural number, and P represents the total number of pixels in the corresponding optimized channel image.

4. The reversing image optimization and warning system according to claim 3, characterized in that: The process of filtering the channel images corresponding to each color channel in the corresponding HSV space to obtain the corresponding optimized channel images includes: Performing bilateral filtering on the channel image to obtain a corresponding first filtered image; At the same time, the channel image is segmented to obtain P0 non-overlapping image blocks; two image blocks are read arbitrarily and recorded as p1 and p2, and the similarity distance between the corresponding image blocks is obtained. In the formula, TX p1 and TX p2 Respectively represent the pixel values ​​of the central pixels in the read image blocks p1 and p2; M and N represent the length and width of the channel image respectively; ‖.‖ represents the Euclidean distance operation; where P0 is a constant; Set a distance threshold, and compare the obtained similarity distance with the distance threshold; if the similarity distance is less than or equal to the distance threshold, no other operations are performed; if the similarity distance is greater than the distance threshold, it indicates that the image blocks p1 and p2 are similar image blocks, and they are divided into the same block group; Based on the above block grouping division process, all image blocks are divided to obtain a plurality of image block groups; Get the image blocks in the image group and perform image superposition on them to obtain a three-dimensional image matrix JZ c Among them, JZ c represents the three-dimensional image matrix corresponding to the c-th image group; c = 1, 2, ..., B; B is a natural number, and B represents the total number of image groups; Perform matrix transformation on the obtained three-dimensional image matrix to obtain the image estimation matrix of the corresponding image group; the formula for matrix transformation is: In the formula, and Γ 3D () represent the three-dimensional inverse transform and three-dimensional transform respectively; ξ represents the hard threshold filtering operation; Represents the three-dimensional image matrix JZ c The corresponding image estimation matrix; obtaining the corresponding estimated area image based on the image estimation matrix; Performing weighted summation on the image block and the corresponding estimated region image to obtain a corresponding optimized image block, and replacing the original image block in the corresponding channel image; Based on the acquisition process of the corresponding optimized image blocks, all image blocks in the channel image are traversed, and the optimized image blocks are used to replace the original image blocks. After the replacement is completed, the channel image after the replacement is recorded as the second filtered image; The obtained first filtered image and the second filtered image are weightedly fused to obtain the corresponding optimized channel image; wherein the weighted fusion formula is: TX yh =ω×TX lb1 +(1-ω)TX lb2 ; In the formula, TX yh Indicates optimized channel image; TX lb1 and TX lb2 represent the first filtered image and the second filtered image respectively; ω represents the fusion weight corresponding to the first filtered image.

5. The reversing image optimization and warning system according to claim 4, characterized in that: The formula for bilateral filtering is: In the formula, TX dl (a, b) represents the pixel value at the pixel point (a, b) in the first filtered image; Z represents the filter size selected in the corresponding bilateral filtering process; TX td (m,n) represents the pixel value of the pixel at coordinate (m,n) adjacent to the pixel (a,b) in the channel image; W(a,b,m,n) represents the weight coefficient of the filter; where W(a,b,m,n)=U(a,b,m,n)×R(a,b,m,n); where U(a,b,m,n) and R(a,b,m,n) represent the spatial kernel function and the color kernel function respectively; where; Where F(a,b) represents the pixel value at the pixel point (a,b) in the channel image; ε R and ε U Represent the spatial proximity factor and color similarity factor respectively.

6. The reversing image optimization and warning system according to claim 5, characterized in that: The process of performing collision risk monitoring on the reversing process of the target vehicle based on the optimized image sequence and obtaining corresponding monitoring results includes: Obtain the median value of the pixel values ​​corresponding to the last row of pixel points in the corresponding optimized reversing image, and record it as the road surface reference pixel value. At the same time, mark the pixel point corresponding to the corresponding median value as the road surface reference point; The Euclidean distances between other pixel points in the optimized reversing image and the corresponding road reference points are respectively obtained, and based on the Euclidean distances, pixel point estimation is performed on other pixel points in the corresponding optimized reversing image; and the corresponding estimated pixel values ​​are obtained; The obtained estimated pixel value is calculated to obtain a corresponding pixel deviation by comparing the pixel value of the corresponding pixel point in the optimized reversing image; A deviation threshold is set, and the obtained pixel deviation is compared with the corresponding deviation threshold. If the pixel deviation is less than or equal to the deviation threshold, the corresponding pixel point is marked as a road surface similarity point; if the pixel deviation is greater than the deviation threshold, the corresponding pixel point is marked as an object suspected point; Based on the suspected points of the object and in combination with the contour extraction technology, the corresponding suspected contour of the object is obtained; and the circumscribed rectangle of the corresponding suspected contour of the object is marked as the suspected area of ​​the object; Obtain pixel coordinates of diagonal pixels in the suspected object area, and based on them, obtain the pixel height of the corresponding suspected object area; and compare it with the preset height threshold; if the pixel height is less than the pixel threshold, no other operations are performed; if the pixel height is not less than the height threshold, the corresponding suspected object area is marked as an obstacle area; Obtain pixel coordinates in the corresponding obstacle area and perform coordinate transformation on them to obtain obstacle coordinate information of the obstacle object corresponding to the corresponding obstacle area in the world coordinate system; At the same time, the three-dimensional coordinates of the camera on the corresponding vehicle at the time when the corresponding optimized reversing image is located are obtained; and the first safety distance is obtained based on the three-dimensional coordinates of the camera and the obstacle coordinate information; At the same time, ultrasonic ranging is performed based on the target vehicle's onboard radar and combined with the obstacle coordinate information to obtain the corresponding second safety distance; Then, the vehicle speed of the target vehicle is read, and the safety distance of the next frame of the optimized reversing image is predicted in combination with the first safety distance and the second safety distance, and the safety distance is marked as the target safety distance, and the corresponding target safety distance is marked as a distance in the read optimized reversing image; After the distance marking is completed, the optimized image sequence for distance marking is converted into a video format, and displayed to the vehicle user based on the on-board display of the target vehicle; At the same time; set the collision range, and take corresponding early warning measures based on the collision safety range of the target safety distance.

7. The reversing image optimization and warning system according to claim 6, characterized in that: The estimated pixel value Where D g0 represents the Euclidean distance between the g-th pixel point in the optimized reversing image and the corresponding road reference point, where g = 0, 1, 2, ..., G, G represents the total number of pixels in the optimized reversing image; G is a natural number; JD0 represents the road reference pixel value; sp(g) and sz(g) represent the horizontal pixel distance and vertical pixel distance between the pixel point g and the road reference point, respectively; ρ2 and ρ1 represent the distance weights of the vertical pixel distance and the horizontal pixel distance, respectively.

Images