A car rearview mirror display system with built-in camera

Abstract

The present invention provides a car rearview mirror display system with a built-in camera, comprising: a camera device, used to obtain a first video within a corresponding camera range based on a start-up instruction; a main control device, used to analyze a corresponding blind spot orientation based on the first video, and generate a corresponding camera adjustment instruction based on the blind spot orientation; the camera device is also used to obtain a second video within the corresponding camera range based on the camera adjustment instruction; a display device is used to project the second video onto a display screen for display; and the system is used to solve the blind spot problem of the left and right optical rearview mirrors outside the car cab, by having built-in left and right display screens, and installing a high-definition infrared camera lens on the outside of the car body to connect with the rearview mirrors, and the camera range can be automatically adjusted based on the camera device, so as to achieve comprehensive monitoring of the blind spot scenes of the left and right rearview mirrors outside the car cab and display them clearly and immediately.

Description

Technical Field

[0001] The invention relates to the technical field of electronic exterior rearview mirrors, and in particular to a car rearview mirror display system with a built-in camera. Background Art

[0002] The rearview mirror is a tool for the driver to directly obtain external information such as the rear, side and bottom of the car while sitting in the driver's seat. In order to facilitate the driver's operation, prevent traffic accidents and ensure personal safety, all countries have stipulated that rearview mirrors must be installed on cars, and all rearview mirrors must be able to adjust the direction.

[0003] At present, most traditional exterior rearview mirrors are mechanical optical devices fixed on the outside of the vehicle body. Although simple mechanical adjustment actions such as folding and unfolding can be achieved through manually adjusting the mechanical device, multi-angle and multi-range adjustments are still not possible. Moreover, this is not only cumbersome to operate, but also requires the driver to personally observe the conditions outside the vehicle in order to reasonably adjust the rearview mirror. Even so, manual adjustment of the rearview mirror still has adjustment errors, resulting in poor adjustment effects. In addition, the optical reflection range of the rearview mirror is limited, and it is impossible to simultaneously monitor and observe a wide range of angles outside the vehicle. There are blind spots, which cannot meet people's requirements for knowing vehicle driving information while driving.

[0004] Therefore, the present invention provides a car rearview mirror display system with a built-in camera. Summary of the invention

[0005] The present invention provides a car rearview mirror display system with a built-in camera, which is used to solve the blind spot problem of the left and right optical rearview mirrors outside the car cab. By having built-in left and right display screens and installing a high-definition infrared camera lens outside the car body to connect with them, the camera range can be automatically adjusted based on the camera device, so as to achieve comprehensive monitoring of the blind spot scenes of the left and right rearview mirrors outside the car cab and display them clearly and instantly.

[0006] The present invention provides a car rearview mirror display system with a built-in camera, comprising:

[0007] A camera device, used for acquiring a first video within a corresponding camera range based on a start instruction;

[0008] A main control device, configured to analyze a corresponding blind spot position based on the first video, and generate a corresponding camera adjustment instruction based on the blind spot position;

[0009] The camera device is further used to obtain a second video within a corresponding camera range based on the camera adjustment instruction;

[0010] A display device is used to project the second video onto a display screen for display.

[0011] Preferably, the camera device comprises:

[0012] A first receiving module is used to receive a start instruction from a main circuit of the vehicle in real time;

[0013] The camera module is used to obtain a first video within a corresponding camera range based on a high-definition infrared camera arranged outside the vehicle body when the start instruction is received.

[0014] Preferably, the camera module comprises:

[0015] An image acquisition unit, configured to acquire a first image based on a high-definition infrared camera disposed outside the vehicle body when receiving the start instruction;

[0016] a deviation analysis unit, configured to analyze, based on the first image, a first deviation angle and a first deviation distance between a current camera range corresponding to the high-definition infrared camera and a first preset camera range, and determine a corresponding first deviation degree based on the first deviation angle and the first deviation distance;

[0017] a deviation judgment unit, configured to judge whether the first deviation is greater than a first deviation threshold, and if so, generate a corresponding first control instruction based on the first deviation, otherwise, generate a corresponding holding instruction;

[0018] A camera control unit is used to control the high-definition infrared camera to obtain a first video within a corresponding camera range based on the first control instruction or the hold instruction.

[0019] Preferably, the deviation analysis unit comprises:

[0020] A first determining subunit, configured to determine a first area corresponding to a vehicle body target in the first image;

[0021] A second determining subunit is used to retrieve a preset image corresponding to the first preset camera range, and determine a second area corresponding to the vehicle body target in the preset image;

[0022] A deviation determination subunit, configured to determine a first deviation angle and a first deviation distance between the high-definition infrared camera and a first preset camera range based on a second deviation angle and a second deviation distance between the first area and the second area;

[0023] The deviation degree determining subunit is used to determine a corresponding first deviation degree based on the first deviation angle and the first deviation distance.

[0024] Preferably, the deviation judgment unit includes:

[0025] A judging subunit, configured to judge whether the first deviation is greater than a first deviation threshold;

[0026] a first generating subunit, configured to determine a corresponding third deviation angle and a third deviation distance based on the first deviation degree and the first deviation degree threshold when the first deviation degree is greater than the first deviation degree threshold, and to generate a corresponding first control instruction based on the third deviation angle and the third deviation distance;

[0027] The second generating subunit is configured to generate a corresponding holding instruction when the first deviation degree is not greater than the first deviation degree threshold.

[0028] Preferably, the main control device comprises:

[0029] A target determination module, used to determine the vehicle body target, external static target and external dynamic target contained in the first video;

[0030] A direction analysis module, used for analyzing the corresponding blind spot direction based on the vehicle body target, the external static target and the external dynamic target;

[0031] An instruction generation module is used to generate corresponding camera adjustment instructions based on the blind spot orientation.

[0032] Preferably, the target determination module includes:

[0033] A first determining unit is used to determine the same area contained in each frame of the second image in the first video, and use the same area as the vehicle body target in the first video;

[0034] A second determination unit is used to obtain a central driving trajectory and a real-time driving speed of the vehicle body, calculate a corresponding first real-time imaging scaling ratio based on the driving speed, and simulate a corresponding first imaging displacement trajectory based on the central driving trajectory;

[0035] A third determining unit, configured to determine a first area other than the same area in the second image;

[0036] A color block recognition unit, used to perform color block recognition on the first area to obtain a corresponding recognition result;

[0037] A region division unit, configured to divide the first region into a plurality of recognition targets based on the recognition result;

[0038] a fourth determining unit, configured to determine, in the first area, an identification target area corresponding to the identification target;

[0039] A region sorting unit, configured to sort the identified target regions based on a frame sequence of all first regions in the first video, and obtain a region tracking result corresponding to the identified target;

[0040] A fifth determining unit, configured to determine a second real-time imaging scaling ratio and a second imaging displacement trajectory corresponding to the identified target based on the area tracking result;

[0041] a matching degree calculation unit, configured to calculate a first matching degree between the first real-time imaging scaling ratio and the second real-time imaging scaling ratio, and at the same time, calculate a second matching degree between the first imaging displacement trajectory and the second imaging displacement trajectory;

[0042] A first discrimination unit, configured to take the identified target whose first matching degree is greater than a first threshold and whose second matching degree is greater than a second threshold as an external static target contained in the first video;

[0043] The second discrimination unit is used to take the identified target whose first matching degree is not greater than a first threshold or whose second matching degree is not greater than a second threshold as an external dynamic target included in the first video.

[0044] Preferably, the orientation analysis module comprises:

[0045] a trajectory determination unit, configured to determine, based on the first video, a first displacement trajectory of the vehicle body target in the first video, a second displacement trajectory of the external static target in the first video, and a third displacement trajectory of the external dynamic target in the first video;

[0046] a first calculation unit, configured to calculate a first intersection probability between the first displacement trajectory and the second displacement trajectory, and at the same time, calculate a second intersection probability between the first displacement trajectory and the third displacement trajectory;

[0047] an image dividing unit, configured to divide the third image into a plurality of adjustment direction areas corresponding to each adjustment direction included in the adjustment direction list according to the adjustment direction list;

[0048] A second calculation unit is used to calculate the area ratio of the sub-area of ​​each identified target in the adjustment direction area to the total area of ​​the corresponding identified target in the third image;

[0049] A third calculation unit, configured to calculate an adjustment weight corresponding to the adjustment direction area based on an area proportion corresponding to the identification target contained in the adjustment direction area and a corresponding intersection probability;

[0050] A weight sorting unit is used to sort all the adjusted weights in descending order to obtain a corresponding weight sorting sequence;

[0051] A fourth calculation unit, used to calculate the weight difference between adjacent adjustment weights included in the weight sorting sequence;

[0052] A weight binding unit, used to bind adjacent weight differences whose weight differences are less than a weight difference threshold to obtain a corresponding binding result;

[0053] an area determination unit, configured to determine a corresponding maximum adjustment weight binding area based on the binding result and the maximum adjustment weight included in the weight sorting sequence;

[0054] The orientation judgment unit is used to judge whether there is only one adjustment direction area contained in the maximum adjustment weight binding area. If so, the adjustment direction corresponding to the corresponding adjustment direction area is used as the corresponding blind spot orientation; otherwise, it is judged whether the maximum interval angle between the adjustment direction areas contained in the maximum adjustment weight binding area is greater than the interval angle threshold. If so, the current camera angle is used as the corresponding blind spot orientation; otherwise, the center orientation of all adjustment direction areas contained in the maximum adjustment weight binding area is used as the corresponding blind spot orientation.

[0055] Preferably, the camera device further includes:

[0056] A second receiving module, used for receiving the camera adjustment instruction in real time;

[0057] The camera control module is used to control the high-definition infrared camera to adjust the corresponding camera range based on the camera adjustment instruction to obtain a second video within the corresponding camera range.

[0058] Preferably, the display device comprises:

[0059] A video receiving module, used for receiving the second video in real time;

[0060] A parameter determination module, used to determine corresponding delivery display parameters based on the camera angle corresponding to the second video and the placement angle of the display screen;

[0061] The display control module is used to adjust the second video based on the delivery display parameters, obtain a corresponding adjusted video, and deliver the adjusted video to a display screen for display.

[0062] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description, claims, and drawings.

[0063] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0065] Figure 1 This is a schematic diagram of a car rearview mirror display system with a built-in camera in an embodiment of the present invention;

[0066] Figure 2 This is an intention of a camera device in an embodiment of the present invention;

[0067] Figure 3 This is a camera module according to an embodiment of the present invention;

[0068] Figure 4 This is a deviation analysis unit in an embodiment of the present invention;

[0069] Figure 5 This is an intention of a deviation judgment unit in an embodiment of the present invention;

[0070] Figure 6 This is an intention of a main control device in an embodiment of the present invention;

[0071] Figure 7 An intention of a target determination module in an embodiment of the present invention;

[0072] Figure 8 This is an intention of a position analysis module in an embodiment of the present invention;

[0073] Fig. 9 This is another camera device according to an embodiment of the present invention;

[0074] Fig.10 This is a display device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0075] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0076] Embodiment 1:

[0077] The present invention provides a car rearview mirror display system with a built-in camera, referring to Figure 1 ,include:

[0078] A camera device, used for acquiring a first video within a corresponding camera range based on a start instruction;

[0079] A main control device, configured to analyze a corresponding blind spot position based on the first video, and generate a corresponding camera adjustment instruction based on the blind spot position;

[0080] The camera device is further used to obtain a second video within a corresponding camera range based on the camera adjustment instruction;

[0081] A display device is used to project the second video onto a display screen for display.

[0082] In this embodiment, the start instruction is an instruction issued based on the start of the circuit of the vehicle body, and is also an instruction for controlling the camera device to start shooting.

[0083] In this embodiment, the first video is the video within the corresponding camera range obtained based on the start instruction.

[0084] In this embodiment, the blind spot position is the position that needs to be focused on after the next adjustment based on the analysis of the first video.

[0085] In this embodiment, the camera adjustment instruction is a control instruction for adjusting the camera range to the corresponding blind spot range based on the blind spot position.

[0086] In this embodiment, the second video is a video obtained after adjusting the camera range of the camera device based on the camera adjustment instruction.

[0087] In this embodiment, the left and right display screens are display screens arranged inside the vehicle body.

[0088] The beneficial effects of the above technology are: it solves the blind spot problem of the left and right optical rearview mirrors on the outside of the car's cab, through built-in left and right display screens, and installing a high-definition infrared camera lens on the outside of the car body to connect with them, it can realize automatic adjustment of the camera range based on the camera device, and realize comprehensive monitoring of the blind spot scenes of the left and right rearview mirrors on the outside of the car's cab and display them clearly and instantly.

[0089] Embodiment 2:

[0090] Based on Example 1, the camera device, referring to Figure 2 ,include:

[0091] A first receiving module is used to receive a start instruction from a main circuit of the vehicle in real time;

[0092] The camera module is used to obtain a first video within a corresponding camera range based on a high-definition infrared camera arranged outside the vehicle body when the start instruction is received.

[0093] In this embodiment, the automobile main body circuit is a circuit for controlling the start of the automobile.

[0094] The beneficial effects of the above technology are: based on the startup instructions of the car's main circuit and the high-definition infrared camera set outside the car body, the first video within the corresponding camera range is obtained, and the camera device can be automatically turned on when the car starts. The high-definition infrared camera has a clearer and wider-angle imaging picture than the traditional rearview mirror.

[0095] Embodiment 3:

[0096] On the basis of Embodiment 2, the camera module, referring to Figure 3 ,include:

[0097] An image acquisition unit, configured to acquire a first image based on a high-definition infrared camera disposed outside the vehicle body when receiving the start instruction;

[0098] a deviation analysis unit, configured to analyze, based on the first image, a first deviation angle and a first deviation distance between a current camera range corresponding to the high-definition infrared camera and a first preset camera range, and determine a corresponding first deviation degree based on the first deviation angle and the first deviation distance;

[0099] a deviation judgment unit, configured to judge whether the first deviation is greater than a first deviation threshold, and if so, generate a corresponding first control instruction based on the first deviation, otherwise, generate a corresponding holding instruction;

[0100] A camera control unit is used to control the high-definition infrared camera to obtain a first video within a corresponding camera range based on the first control instruction or the hold instruction.

[0101] In this embodiment, the first image is an image acquired by a high-definition infrared camera disposed outside the vehicle body when the start instruction is received.

[0102] In this embodiment, the first deviation angle is a deviation angle between the current camera range analyzed based on the first image and the first preset camera range.

[0103] In this embodiment, the first preset camera range is a preset imaging range that can analyze the deviation angle and deviation distance of the high-definition infrared camera.

[0104] In this embodiment, the current camera range is the current camera range of the high-definition infrared camera.

[0105] In this embodiment, the first deviation distance is the deviation distance between the current camera range analyzed based on the first image and the first preset camera range.

[0106] In this embodiment, the first deviation is a two-dimensional array based on the first deviation angle and the first deviation distance, and is also a value representing the deviation between the current camera range and the first preset camera range.

[0107] In this embodiment, judging whether the first deviation degree is greater than a first deviation degree threshold includes: when the first deviation angle in the first deviation degree is greater than the first deviation angle threshold in the first deviation degree threshold and when the first deviation distance in the first deviation degree is greater than the first deviation distance threshold in the first deviation degree threshold, then judging that the first deviation degree is greater than the first deviation degree threshold; otherwise, judging that the first deviation degree is not greater than the first deviation degree threshold.

[0108] In this embodiment, the first deviation threshold is the minimum first deviation corresponding to when the high-definition infrared camera is determined to need to be adjusted based on the corresponding first control instruction.

[0109] In this embodiment, the first control instruction is a control instruction generated based on the first deviation and used to control the high-definition infrared camera to make adjustments.

[0110] In this embodiment, the hold instruction is an instruction for controlling the high-definition infrared camera to maintain the current state.

[0111] The beneficial effects of the above technology are: based on the deviation between the current camera range corresponding to the high-definition infrared camera in the first image obtained when the start-up command is received and the first preset camera range, it can be determined whether the current camera angle and camera range of the infrared high-definition camera need to be adjusted, thereby realizing automatic adjustment of the infrared high-definition camera when the car is started, ensuring that good imaging effects are presented in time after the car is started, and providing an imaging basis for the subsequent analysis of the blind spot orientation.

[0112] Embodiment 4:

[0113] On the basis of Example 3, the deviation analysis unit, referring to Figure 4 ,include:

[0114] A first determining subunit, configured to determine a first area corresponding to a vehicle body target in the first image;

[0115] A second determining subunit is used to retrieve a preset image corresponding to the first preset camera range, and determine a second area corresponding to the vehicle body target in the preset image;

[0116] A deviation determination subunit, configured to determine a first deviation angle and a first deviation distance between the high-definition infrared camera and a first preset camera range based on a second deviation angle and a second deviation distance between the first area and the second area;

[0117] The deviation degree determining subunit is used to determine a corresponding first deviation degree based on the first deviation angle and the first deviation distance.

[0118] In this embodiment, the first area is the area corresponding to the vehicle body target contained in the first image.

[0119] In this embodiment, the vehicle body target is the main vehicle on which the vehicle-mounted intelligent electronic exterior rearview mirror with built-in left and right display screens is installed.

[0120] In this embodiment, the preset image is an image including the vehicle body corresponding to when the high-definition infrared camera is within the first preset camera range.

[0121] In this embodiment, the second area is the area of ​​the vehicle body target in the preset image.

[0122] In this embodiment, the second deviation angle is the angle between a line between the center of the first area and the center of the second area and the vertical axis.

[0123] In this embodiment, the second deviation distance is the distance between the center of the first area and the center of the second area.

[0124] In this embodiment, the first deviation angle and the first deviation distance between the high-definition infrared camera and the first preset camera range are determined based on the second deviation angle and the second deviation distance between the first area and the second area, that is: the first deviation angle is regarded as the corresponding first deviation angle, and the second deviation distance is regarded as the corresponding first deviation distance.

[0125] In this embodiment, determining a corresponding first deviation degree based on the first deviation angle and the first deviation distance includes:

[0126]

[0127] Wherein, ρ is the first deviation degree, e is a natural constant and the value of e is 2.72, ε1 is the first deviation angle, ε0 is the maximum deviation angle and the value of ε0 is 90 degrees, L1 is the first deviation distance, L0 is the maximum deviation distance and the value of L0 is the diagonal length of the first image;

[0128] For example, if ε1 is 45 degrees, L1 is 10, and L0 is 100, then ρ is 0.74082.

[0129] The beneficial effect of the above technology is: by calculating the deviation distance and deviation angle between the first area corresponding to the vehicle body in the first image corresponding to the current camera range corresponding to the high-definition infrared camera and the second area corresponding to the vehicle body in the preset image within the first preset camera range, the corresponding deviation degree is calculated, which provides a data basis for subsequent judgment on whether the current camera angle and camera range of the infrared high-definition camera need to be adjusted.

[0130] Embodiment 5:

[0131] On the basis of Example 4, the deviation judgment unit refers to Figure 5 ,include:

[0132] A judging subunit, configured to judge whether the first deviation is greater than a first deviation threshold;

[0133] a first generating subunit, configured to determine a corresponding third deviation angle and a third deviation distance based on the first deviation degree and the first deviation degree threshold when the first deviation degree is greater than the first deviation degree threshold, and to generate a corresponding first control instruction based on the third deviation angle and the third deviation distance;

[0134] The second generating subunit is configured to generate a corresponding holding instruction when the first deviation degree is not greater than the first deviation degree threshold.

[0135] In this embodiment, the third deviation angle is the angle difference between the first deviation angle included in the first deviation degree and the deviation angle threshold included in the first deviation degree threshold.

[0136] In this embodiment, the third deviation distance is the distance difference between the first deviation distance included in the first deviation degree and the deviation distance threshold value included in the first deviation degree threshold value.

[0137] The beneficial effects of the above technology are: based on the difference between the deviation distance and deviation angle contained in the first deviation degree and the deviation angle threshold and deviation distance threshold contained in the first deviation degree threshold, the corresponding third deviation angle and third deviation distance can be calculated, and then based on the third deviation angle and the third deviation threshold, it can be determined whether the current shooting angle and shooting range of the infrared high-definition camera need to be adjusted, and then the corresponding first control instruction or hold instruction is generated.

[0138] Embodiment 6:

[0139] On the basis of Example 5, the main control device, referring to Figure 6 ,include:

[0140] A target determination module, used to determine the vehicle body target, external static target and external dynamic target contained in the first video;

[0141] A direction analysis module, used for analyzing the corresponding blind spot direction based on the vehicle body target, the external static target and the external dynamic target;

[0142] An instruction generation module is used to generate corresponding camera adjustment instructions based on the blind spot orientation.

[0143] In this embodiment, the external static target is a target contained in the first video that is absolutely still relative to the ground.

[0144] In this embodiment, the external dynamic target is a target included in the first video and moving absolutely relative to the ground.

[0145] The beneficial effect of the above technology is: based on the analysis of the vehicle body target, external static target and external dynamic target contained in the first video, it is possible to identify the directions where potential dangerous factors exist in the driving process of the current vehicle body in the current environment and which cannot be monitored by the current camera range.

[0146] Embodiment 7:

[0147] On the basis of Example 6, the target determination module refers to Figure 7 ,include:

[0148] A first determining unit is used to determine the same area contained in each frame of the second image in the first video, and use the same area as the vehicle body target in the first video;

[0149] A second determination unit is used to obtain a central driving trajectory and a real-time driving speed of the vehicle body, calculate a corresponding first real-time imaging scaling ratio based on the driving speed, and simulate a corresponding first imaging displacement trajectory based on the central driving trajectory;

[0150] A third determining unit, configured to determine a first area other than the same area in the second image;

[0151] A color block recognition unit, used to perform color block recognition on the first area to obtain a corresponding recognition result;

[0152] A region division unit, configured to divide the first region into a plurality of recognition targets based on the recognition result;

[0153] a fourth determining unit, configured to determine, in the first area, an identification target area corresponding to the identification target;

[0154] A region sorting unit, configured to sort the identified target regions based on a frame sequence of all first regions in the first video, and obtain a region tracking result corresponding to the identified target;

[0155] A fifth determining unit, configured to determine a second real-time imaging scaling ratio and a second imaging displacement trajectory corresponding to the identified target based on the area tracking result;

[0156] a matching degree calculation unit, configured to calculate a first matching degree between the first real-time imaging scaling ratio and the second real-time imaging scaling ratio, and at the same time, calculate a second matching degree between the first imaging displacement trajectory and the second imaging displacement trajectory;

[0157] A first discrimination unit, configured to take the identified target whose first matching degree is greater than a first threshold and whose second matching degree is greater than a second threshold as an external static target contained in the first video;

[0158] The second discrimination unit is used to take the identified target whose first matching degree is not greater than a first threshold or whose second matching degree is not greater than a second threshold as an external dynamic target included in the first video.

[0159] In this embodiment, the second image is a video frame included in the first video.

[0160] In this embodiment, the same region is the same image region in all the second images.

[0161] In this embodiment, the central driving trajectory is the driving trajectory of the center of the vehicle body.

[0162] In this embodiment, the real-time driving speed is the driving speed of the vehicle body obtained in real time.

[0163] In this embodiment, calculating the corresponding first real-time imaging scaling ratio based on the driving speed includes:

[0164] δ=vtεθ

[0165] Wherein, δ is the first real-time imaging scaling ratio, v is the driving speed, t is the unit time, ε is the coefficient of the relationship between the imaging ratio and the distance, and the unit of ε is / meter, θ is the initial imaging ratio of the high-definition infrared camera;

[0166] For example, if v is 100, t is 1, ε is 0.1, and θ is 0.01, the first real-time imaging scaling factor is 0.1.

[0167] In this embodiment, the corresponding first imaging displacement trajectory is simulated based on the central driving trajectory, that is, the displacement trajectory of the stationary object in the current camera range in the video frame included in the first video is simulated by using the reverse direction trajectory of the central driving trajectory and the imaging parameters.

[0168] In this embodiment, the first region is the region other than the same region in the second image.

[0169] In this embodiment, the recognition result is the result obtained by performing color block recognition on the first area.

[0170] In this embodiment, the color block recognition is to divide the first area into a plurality of sub-areas corresponding to different chromaticity value ranges according to a preset chromaticity value gradient range.

[0171] In this embodiment, the recognition target is to divide the first area into multiple image areas based on the recognition result.

[0172] In this embodiment, the region tracking result is the image region tracking result of the corresponding recognition target in the first video obtained by sorting the corresponding recognition target regions based on the frame sequence of all the first regions in the first video.

[0173] In this embodiment, a second real-time imaging scaling ratio and a second imaging displacement trajectory corresponding to the identified target are determined based on the regional tracking result, including: calculating the ratio of the regional area of ​​the identified target in the first video in the current video frame to the regional area in the previous video frame by the regional tracking result, obtaining the corresponding second real-time imaging scaling ratio, determining the center point of the identified target in each video frame in the first video based on the regional tracking result, and connecting the center points according to the frame sequence of the regional tracking result to fit the corresponding second imaging displacement trajectory.

[0174] In this embodiment, a first matching degree between the first real-time imaging scaling ratio and the second real-time imaging scaling ratio is calculated, that is,

[0175] The ratio of the second real-time imaging scaling ratio to the first real-time imaging scaling ratio is used as the corresponding first matching degree.

[0176] In this embodiment, the second matching degree between the first imaging displacement trajectory and the second imaging displacement trajectory is calculated, that is,

[0177] The ratio of the second imaging displacement trajectory to the first imaging displacement trajectory is used as the corresponding second matching degree.

[0178] In this embodiment, the first threshold is the minimum first matching degree corresponding to when the identified target is determined to be an external static target.

[0179] In this embodiment, the second threshold is the minimum second matching degree corresponding to when the identified target is determined to be an external static target.

[0180] The beneficial effects of the above technology are: based on tracking the trajectories and imaging scale changes of all color block areas contained in the first video, corresponding tracking results are obtained, and the imaging displacement trajectory and imaging scale determined based on the driving speed of the vehicle body are compared with the corresponding results in the tracking results, thereby realizing the identification of the vehicle body target and external static targets and external dynamic targets contained in the first video based on the corresponding relationship between the motion state of the background objects in the environment and the imaging results, which provides an important basis for subsequently determining the blind spot orientation of the current camera range.

[0181] Embodiment 8:

[0182] Based on Example 7, the orientation analysis module refers to Figure 8 ,include:

[0183] a trajectory determination unit, configured to determine, based on the first video, a first displacement trajectory of the vehicle body target in the first video, a second displacement trajectory of the external static target in the first video, and a third displacement trajectory of the external dynamic target in the first video;

[0184] a first calculation unit, configured to calculate a first intersection probability between the first displacement trajectory and the second displacement trajectory, and at the same time, calculate a second intersection probability between the first displacement trajectory and the third displacement trajectory;

[0185] an image dividing unit, configured to divide the third image into a plurality of adjustment direction areas corresponding to each adjustment direction included in the adjustment direction list according to the adjustment direction list;

[0186] A second calculation unit is used to calculate the area ratio of the sub-area of ​​each identified target in the adjustment direction area to the total area of ​​the corresponding identified target in the third image;

[0187] A third calculation unit, configured to calculate an adjustment weight corresponding to the adjustment direction area based on an area proportion corresponding to the identification target contained in the adjustment direction area and a corresponding intersection probability;

[0188] A weight sorting unit is used to sort all the adjusted weights in descending order to obtain a corresponding weight sorting sequence;

[0189] A fourth calculation unit, used to calculate the weight difference between adjacent adjustment weights included in the weight sorting sequence;

[0190] A weight binding unit, used to bind adjacent weight differences whose weight differences are less than a weight difference threshold to obtain a corresponding binding result;

[0191] an area determination unit, configured to determine a corresponding maximum adjustment weight binding area based on the binding result and the maximum adjustment weight included in the weight sorting sequence;

[0192] The orientation judgment unit is used to judge whether there is only one adjustment direction area contained in the maximum adjustment weight binding area. If so, the adjustment direction corresponding to the corresponding adjustment direction area is used as the corresponding blind spot orientation; otherwise, it is judged whether the maximum interval angle between the adjustment direction areas contained in the maximum adjustment weight binding area is greater than the interval angle threshold. If so, the current camera angle is used as the corresponding blind spot orientation; otherwise, the center orientation of all adjustment direction areas contained in the maximum adjustment weight binding area is used as the corresponding blind spot orientation.

[0193] In this embodiment, the first displacement trajectory is the displacement trajectory of the vehicle body target in the first video determined based on the first video.

[0194] In this embodiment, the second displacement trajectory is the displacement trajectory of the external static target in the first video determined based on the first video.

[0195] In this embodiment, the third displacement trajectory is the displacement trajectory of the external dynamic target in the first video determined based on the first video.

[0196] In this embodiment, calculating a first intersection probability of the first displacement trajectory and the second displacement trajectory includes:

[0197] A first tangent angle at the end point of the first displacement trajectory and a second tangent angle at the end point of the second displacement trajectory are determined, and a ratio of a difference between the first defect angle and the second tangent angle and 90 degrees is used as a corresponding first intersection probability.

[0198] In this embodiment, calculating the second intersection probability of the first displacement trajectory and the third displacement trajectory includes:

[0199] A first tangent angle at the end point of the first displacement trajectory and a third tangent angle at the end point of the third displacement trajectory are determined, and a ratio of a difference between the first defect angle and the third tangent angle to 90 degrees is used as a corresponding second intersection probability.

[0200] In this embodiment, the adjustment direction list can be specifically obtained according to the mechanical setting device of the infrared high-definition camera, for example: it includes eight adjustment directions, which are (0, 45] degrees, (45, 90] degrees, (90, 135] degrees, (135, 180] degrees, (180, 225] degrees, (225, 270] degrees, (270, 315] degrees, and (315, 360] degrees.

[0201] In this embodiment, the adjustment direction area is to divide the third image into multiple image areas corresponding to each adjustment direction included in the adjustment direction list according to the adjustment direction list. For example, the third image is divided into corresponding multiple image areas with the center point of the third image as the center and the angle range corresponding to each adjustment direction.

[0202] In this embodiment, the area ratio is the ratio of the sub-area corresponding to the recognition target in the adjustment direction area to the total area corresponding to the recognition target in the third image.

[0203] In this embodiment, the sub-region is the region occupied by the recognition target in the adjustment direction region.

[0204] In this embodiment, the total area is the area occupied by the recognition target in the third image.

[0205] In this embodiment, based on the area proportion corresponding to the identification targets contained in the adjustment direction area and the corresponding intersection probability, the adjustment weight corresponding to the adjustment direction area is calculated, that is: the weight corresponding to the corresponding identification target is calculated by multiplying the area proportion corresponding to each identification target and the intersection probability, and the sum of the weights corresponding to all identification targets contained in the adjustment direction area is used as the corresponding adjustment weight.

[0206] In this embodiment, the weight sorting sequence is a sequence obtained by sorting all adjustment weights in descending order.

[0207] In this embodiment, the weight difference is the difference between adjacent adjustment weights included in the weight sorting sequence.

[0208] In this embodiment, the binding result is the result obtained after binding adjacent weight differences whose weight differences are less than the weight difference threshold.

[0209] In this embodiment, the weight difference threshold is the maximum weight difference corresponding to when adjacent weight values ​​are determined to need to be bound.

[0210] In this embodiment, the maximum adjustment weight binding area is an area determined based on the binding result and the maximum adjustment weight included in the weight sorting sequence.

[0211] In this embodiment, the maximum adjustment weight is the maximum adjustment weight value included in the weight sorting sequence.

[0212] In this embodiment, the maximum interval angle is the maximum value of the interval angles between the adjustment direction areas included in the maximum adjustment weight binding area.

[0213] In this embodiment, the interval angle threshold is the minimum value allowed for the maximum value of the interval angle between the adjustment direction areas included in the maximum adjustment weight binding area when the current camera angle is used as the corresponding blind spot orientation.

[0214] In this embodiment, the central orientation is, for example: if all adjustment direction areas included in the maximum adjustment weight binding area are (0, 45] degrees and (45, 90] degrees, 45 degrees is used as the corresponding central orientation.

[0215] The beneficial effects of the above technology are: based on a preset adjustment direction list, the third image is divided into multiple adjustment direction areas, and the corresponding adjustment weights are calculated by the intersection probability and area proportion corresponding to the identification targets contained in the adjustment direction areas, and the adjustment direction areas are bound and judged based on the adjustment weights to determine the corresponding adjustment directions, and the corresponding blind spot orientations are determined based on the adjustment directions, so that the determined blind spot orientations can not only realize the prediction of potential dangerous factors that may cause collisions during vehicle driving, but also achieve a wider range of imaging effects based on the prediction results.

[0216] Embodiment 9:

[0217] Based on Example 8, the camera device, referring to Fig. 9 , also includes:

[0218] A second receiving module, used for receiving the camera adjustment instruction in real time;

[0219] The camera control module is used to control the high-definition infrared camera to adjust the corresponding camera range based on the camera adjustment instruction to obtain a second video within the corresponding camera range.

[0220] The beneficial effects of the above technology are: based on the camera adjustment instruction, the high-definition infrared camera is controlled to adjust the corresponding camera range, and then a second video within the corresponding camera range is obtained, thereby realizing comprehensive monitoring of the blind spots of the left and right rearview mirrors outside the car's cab.

[0221] Embodiment 10:

[0222] On the basis of Example 9, the display device, referring to Fig.10 ,include:

[0223] A video receiving module, used for receiving the second video in real time;

[0224] A parameter determination module, used to determine corresponding delivery display parameters based on the camera angle corresponding to the second video and the placement angle of the display screen;

[0225] The display control module is used to adjust the second video based on the delivery display parameters, obtain a corresponding adjusted video, and deliver the adjusted video to a display screen for display.

[0226] In this embodiment, the delivery display parameters are display setting parameters corresponding to when the second video is delivered to the corresponding left and right display screens.

[0227] In this embodiment, the adjusted video is a video obtained by adjusting the second video based on the delivery display parameters.

[0228] The beneficial effect of the above technology is: to realize the clear and real-time display of the comprehensive monitoring video of the blind spots of the left and right rearview mirrors outside the car's cab.

[0229] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A car rearview mirror display system with a built-in camera, characterized in that: include: A camera device, used for acquiring a first video within a corresponding camera range based on a start instruction; A main control device, configured to analyze a corresponding blind spot position based on the first video, and generate a corresponding camera adjustment instruction based on the blind spot position; The camera device is further used to obtain a second video within a corresponding camera range based on the camera adjustment instruction; A display device, used for projecting the second video onto a display screen for display; Wherein, based on the deviation between the current camera range corresponding to the high-definition infrared camera in the first image obtained when the start command is received and the first preset camera range, it is determined whether the current camera angle and camera range of the infrared high-definition camera need to be adjusted; Calculate the corresponding deviation degree by calculating the deviation distance and deviation angle between the first area corresponding to the vehicle body in the first image corresponding to the current camera range corresponding to the high-definition infrared camera and the second area corresponding to the preset image corresponding to the vehicle body within the first preset camera range; Calculate a corresponding third deviation angle and a third deviation distance based on the difference between the deviation distance and the deviation angle included in the first deviation degree and the deviation angle threshold and the deviation distance threshold included in the first deviation degree threshold, and then determine whether the current camera angle and camera range of the infrared high-definition camera need to be adjusted based on the third deviation angle and the third deviation threshold, and then generate a corresponding first control instruction or a holding instruction; Based on the analysis of the vehicle main body target, the external static target and the external dynamic target contained in the first video, identifying the position where the current vehicle main body has potential dangerous factors during driving in the current environment and which cannot be monitored by the current camera range; Based on tracking the trajectories and imaging scale changes of all color block areas included in the first video, corresponding tracking results are obtained, and the imaging displacement trajectory and imaging scale determined based on the driving speed of the vehicle body are compared with the corresponding results in the tracking results, so as to realize the recognition of the vehicle body target, the external static target, and the external dynamic target included in the first video based on the corresponding relationship between the motion state of the background object in the environment and the imaging result; The third image is divided into multiple adjustment direction areas based on a preset adjustment direction list, and the corresponding adjustment weights are calculated by calculating the intersection probabilities and area proportions corresponding to the identified targets contained in the adjustment direction areas. The adjustment direction areas are bound based on the adjustment weights to determine the corresponding adjustment directions, and the corresponding blind spot positions are determined based on the adjustment directions.

2. The automobile rearview mirror display system with a built-in camera according to claim 1, characterized in that: The camera device comprises: A first receiving module is used to receive a start instruction from a main circuit of the vehicle in real time; The camera module is used to obtain a first video within a corresponding camera range based on a high-definition infrared camera arranged outside the vehicle body when the start instruction is received.

3. The automobile rearview mirror display system with a built-in camera according to claim 2, characterized in that: The camera module comprises: An image acquisition unit, configured to acquire a first image based on a high-definition infrared camera disposed outside the vehicle body when receiving the start instruction; a deviation analysis unit, configured to analyze, based on the first image, a first deviation angle and a first deviation distance between a current camera range corresponding to the high-definition infrared camera and a first preset camera range, and determine a corresponding first deviation degree based on the first deviation angle and the first deviation distance; a deviation judgment unit, configured to judge whether the first deviation is greater than a first deviation threshold, and if so, generate a corresponding first control instruction based on the first deviation, otherwise, generate a corresponding holding instruction; A camera control unit is used to control the high-definition infrared camera to obtain a first video within a corresponding camera range based on the first control instruction or the hold instruction.

4. The automobile rearview mirror display system with a built-in camera according to claim 3, characterized in that: The deviation analysis unit comprises: A first determining subunit, configured to determine a first area corresponding to a vehicle body target in the first image; A second determining subunit is used to retrieve a preset image corresponding to the first preset camera range, and determine a second area corresponding to the vehicle body target in the preset image; A deviation determination subunit, configured to determine a first deviation angle and a first deviation distance between the high-definition infrared camera and a first preset camera range based on a second deviation angle and a second deviation distance between the first area and the second area; The deviation degree determining subunit is used to determine a corresponding first deviation degree based on the first deviation angle and the first deviation distance.

5. The automobile rearview mirror display system with a built-in camera according to claim 4, characterized in that: The deviation judgment unit comprises: A judging subunit, configured to judge whether the first deviation is greater than a first deviation threshold; a first generating subunit, configured to determine a corresponding third deviation angle and a third deviation distance based on the first deviation degree and the first deviation degree threshold when the first deviation degree is greater than the first deviation degree threshold, and to generate a corresponding first control instruction based on the third deviation angle and the third deviation distance; The second generating subunit is configured to generate a corresponding holding instruction when the first deviation degree is not greater than the first deviation degree threshold.

6. The automobile rearview mirror display system with a built-in camera according to claim 5, characterized in that: The main control device comprises: A target determination module, used to determine the vehicle body target, external static target and external dynamic target contained in the first video; A direction analysis module, used for analyzing the corresponding blind spot direction based on the vehicle body target, the external static target and the external dynamic target; An instruction generation module is used to generate corresponding camera adjustment instructions based on the blind spot orientation.

7. The automobile rearview mirror display system with a built-in camera according to claim 6, characterized in that: The target determination module comprises: A first determining unit is used to determine the same area contained in each frame of the second image in the first video, and use the same area as the vehicle body target in the first video; A second determination unit is used to obtain a central driving trajectory and a real-time driving speed of the vehicle body, calculate a corresponding first real-time imaging scaling ratio based on the driving speed, and simulate a corresponding first imaging displacement trajectory based on the central driving trajectory; A third determining unit, configured to determine a first area other than the same area in the second image; A color block recognition unit, used to perform color block recognition on the first area to obtain a corresponding recognition result; A region division unit, configured to divide the first region into a plurality of recognition targets based on the recognition result; a fourth determining unit, configured to determine, in the first area, an identification target area corresponding to the identification target; A region sorting unit, configured to sort the identified target regions based on a frame sequence of all first regions in the first video, and obtain a region tracking result corresponding to the identified target; A fifth determining unit, configured to determine a second real-time imaging scaling ratio and a second imaging displacement trajectory corresponding to the identified target based on the area tracking result; a matching degree calculation unit, configured to calculate a first matching degree between the first real-time imaging scaling ratio and the second real-time imaging scaling ratio, and at the same time, calculate a second matching degree between the first imaging displacement trajectory and the second imaging displacement trajectory; A first discrimination unit, configured to take the identified target whose first matching degree is greater than a first threshold and whose second matching degree is greater than a second threshold as an external static target contained in the first video; The second discrimination unit is used to take the identified target whose first matching degree is not greater than a first threshold or whose second matching degree is not greater than a second threshold as an external dynamic target included in the first video.

8. The automobile rearview mirror display system with a built-in camera according to claim 7, characterized in that: The orientation analysis module comprises: a trajectory determination unit, configured to determine, based on the first video, a first displacement trajectory of the vehicle body target in the first video, a second displacement trajectory of the external static target in the first video, and a third displacement trajectory of the external dynamic target in the first video; a first calculation unit, configured to calculate a first intersection probability between the first displacement trajectory and the second displacement trajectory, and at the same time, calculate a second intersection probability between the first displacement trajectory and the third displacement trajectory; an image dividing unit, configured to divide the third image into a plurality of adjustment direction areas corresponding to each adjustment direction included in the adjustment direction list according to the adjustment direction list; A second calculation unit is used to calculate the area ratio of the sub-area of ​​each identified target in the adjustment direction area to the total area of ​​the corresponding identified target in the third image; A third calculation unit, configured to calculate an adjustment weight corresponding to the adjustment direction area based on an area proportion corresponding to the identification target contained in the adjustment direction area and a corresponding intersection probability; A weight sorting unit is used to sort all the adjusted weights in descending order to obtain a corresponding weight sorting sequence; A fourth calculation unit, used to calculate the weight difference between adjacent adjustment weights included in the weight sorting sequence; A weight binding unit, used to bind adjacent weight differences whose weight differences are less than a weight difference threshold to obtain a corresponding binding result; an area determination unit, configured to determine a corresponding maximum adjustment weight binding area based on the binding result and the maximum adjustment weight included in the weight sorting sequence; The orientation judgment unit is used to judge whether there is only one adjustment direction area contained in the maximum adjustment weight binding area. If so, the adjustment direction corresponding to the corresponding adjustment direction area is used as the corresponding blind spot orientation; otherwise, it is judged whether the maximum interval angle between the adjustment direction areas contained in the maximum adjustment weight binding area is greater than the interval angle threshold. If so, the current camera angle is used as the corresponding blind spot orientation; otherwise, the center orientation of all adjustment direction areas contained in the maximum adjustment weight binding area is used as the corresponding blind spot orientation.

9. The automobile rearview mirror display system with a built-in camera according to claim 8, characterized in that: The camera device further comprises: A second receiving module, used for receiving the camera adjustment instruction in real time; The camera control module is used to control the high-definition infrared camera to adjust the corresponding camera range based on the camera adjustment instruction to obtain a second video within the corresponding camera range.

10. The automobile rearview mirror display system with a built-in camera according to claim 9, characterized in that: The display device comprises: A video receiving module, used for receiving the second video in real time; A parameter determination module, used to determine corresponding delivery display parameters based on the camera angle corresponding to the second video and the placement angle of the display screen; The display control module is used to adjust the second video based on the delivery display parameters, obtain a corresponding adjusted video, and deliver the adjusted video to a display screen for display.

Images