Panoramic head-up display method and device

Abstract

The invention provides a panoramic head-up display method and device, and relates to the field of intelligent interactive driving. The method comprises the following steps: constructing a visual internal coordinate system based on a visual anchor point and a visual cambered surface of a target driver; generating a reference selection box corresponding to a visual target in the head-up display vision field based on the visual internal coordinate system; outputting candidate coordinate points corresponding to the reference selection box in the visual internal coordinate system, and obtaining target coordinate points meeting a preset screening condition from the candidate coordinate points; and constructing a visual display frame corresponding to the visual target based on the target coordinate point, and displaying driving environment information to the target driver in the visual display frame, the driving environment information including road condition information, pedestrian information and driving route information. The problem that in a multi-target dynamic scene of an urban intersection, in a traditional method, a plurality of display frames and prompt elements shield each other, so that key information is shielded by other display content, and the perception experience of head-up display on a driver is seriously affected is solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of intelligent interactive driving, and in particular to a panoramic head-up display method and device. BACKGROUND

[0002] Head-up display (HUD) is also called head-up display system, which displays the related information such as driving speed, navigation, steering and adaptive cruise control by default.

[0003] The traditional head-up display method mainly displays the camera picture, navigation picture, warning information picture and the like by directly superimposing them according to the screen coordinates. This method cannot consider the display influencing factors such as the driver's viewpoint change and the glass curvature, and is prone to offset of the superimposed position with the scene and posture change, so it cannot guarantee the stability and readability of the display content in the panoramic large field of view. Especially in the multi-target dynamic scene of urban intersections, the traditional method is prone to mutual shielding of multiple display boxes and prompt elements, which causes the key information to be shielded by other display content, seriously affecting the perception experience of the driver by the head-up display.

[0004] Therefore, there is an urgent need for a panoramic head-up display method and device. SUMMARY

[0005] The present application provides a panoramic head-up display method and device, which solves the problem that in the multi-target dynamic scene of urban intersections, the traditional method is prone to mutual shielding of multiple display boxes and prompt elements, which causes the key information to be shielded by other display content, seriously affecting the perception experience of the driver by the head-up display.

[0006] In a first aspect of the present application, a panoramic head-up display method is provided, which comprises: constructing a visual internal coordinate system based on the visual anchor point and the visual arc surface of the target driver; generating a reference selection box corresponding to a visual target in the head-up display field of view based on the visual internal coordinate system; outputting the candidate coordinate points corresponding to the reference selection box in the visual internal coordinate system, and obtaining the target coordinate points meeting the preset screening condition from the candidate coordinate points; constructing a visual display box corresponding to the visual target based on the target coordinate points, and displaying the driving environment information to the target driver in the visual display box, wherein the driving environment information includes road condition information, pedestrian information and driving route information.

[0007] Optionally, based on the visual anchor point and the visual arc surface of the target driver, a visual internal coordinate system is constructed, specifically including: confirming the visual anchor point according to the visual pose parameter of the target driver, the visual pose parameter including the head pose parameter, the eye movement fixation point distribution parameter and the seat position parameter; determining the visual arc surface based on the spatial geometric relationship between the visual anchor point and the vehicle windshield; constructing an anisometric coordinate system within the visual range of the target driver based on the visual anchor point and the visual arc surface; constructing the visual internal coordinate system with the visual anchor point as the origin, with the principal tangent and the normal of the visual arc surface as the basis vectors, and with the anisometric coordinate system as the scale reference.

[0008] Optionally, based on the visual internal coordinate system, a reference selection box corresponding to a visual target in the head-up display field of view is generated, specifically including: obtaining the visual target in the head-up display field of view, the visual target including a movable visual target and an immovable visual target; generating an equal-body selection box corresponding to the movable visual target based on the visual internal coordinate system; generating an area selection box corresponding to the immovable visual target based on the visual internal coordinate system.

[0009] Optionally, when the output is the equal-body selection box, candidate coordinate points corresponding to the reference selection box in the visual internal coordinate system are output, specifically including: extracting the four-edge endpoints of the equal-body selection box, and obtaining a first coordinate point set corresponding to the four-edge endpoints in the visual internal coordinate system; taking the first coordinate point set as the candidate coordinate points corresponding to the equal-body selection box.

[0010] Optionally, when the output is the area selection box, candidate coordinate points corresponding to the reference selection box in the visual internal coordinate system are output, specifically including: extracting the four-edge midpoints of the area selection box, and obtaining a second coordinate point set corresponding to the four-edge midpoints in the visual internal coordinate system; taking the second coordinate point set as the candidate coordinate points corresponding to the equal-body selection box.

[0011] Optionally, target coordinate points meeting a preset filtering condition are obtained from the candidate coordinate points, specifically including: calculating the interval between the human-vehicle endpoints and the number of human-vehicle endpoints based on the candidate coordinate points; determining whether the interval between the human-vehicle endpoints is less than a first preset threshold; when it is confirmed that the interval between the human-vehicle endpoints is less than the first preset threshold, the candidate coordinate points are not output; determining whether the interval between the human-vehicle endpoints is greater than a second preset threshold; when it is confirmed that the interval between the human-vehicle endpoints is greater than the second preset threshold, the road endpoints in the candidate coordinate points are excluded, and the remaining coordinate points are output as the target coordinate points, the first preset threshold being less than the second preset threshold; determining whether the number of human-vehicle endpoints is greater than a preset number; when it is confirmed that the number of human-vehicle endpoints is greater than the preset number, the endpoints in the candidate coordinate points with a motion speed greater than a preset speed are output as the target coordinate points.

[0012] Optionally, the interval between the human and vehicle end points and the number of human and vehicle end points are calculated based on the candidate coordinate points, specifically including: marking the candidate coordinate points as human end points and vehicle end points in the visual internal coordinate system according to categories, and pairing each human end point with the nearest vehicle end point according to the nearest field principle to generate human and vehicle end point pairs; calculating the Euclidean distance corresponding to the human and vehicle end point pairs, and calculating the projection distance of the human and vehicle end point pairs along the main tangent of the visual curved surface; weighting the Euclidean distance and the projection distance to obtain the original human and vehicle interval according to a weight, and performing exponential moving average on the original human and vehicle interval according to frames to obtain the human and vehicle end point interval after smoothing processing; performing deduplication statistics on the same type of end points according to a deduplication radius to calculate the number of human end points and the number of vehicle end points in the current frame; and performing summation calculation based on the number of human end points and the number of vehicle end points to output the number of human and vehicle end points.

[0013] In a second aspect of the present application, a panoramic head-up display device is provided, the device comprising an acquisition module and a processing module, wherein, The acquisition module is configured to construct a visual internal coordinate system based on a visual anchor point and a visual curved surface of a target driver, and generate a reference selection box corresponding to a visual target in a head-up display field of view based on the visual internal coordinate system.

[0014] The processing module is configured to output candidate coordinate points corresponding to the reference selection box in the visual internal coordinate system, and acquire target coordinate points meeting a preset screening condition from the candidate coordinate points; construct a visual display box corresponding to the visual target based on the target coordinate points, and display driving environment information to the target driver in the visual display box, the driving environment information including road condition information, pedestrian information, and driving route information.

[0015] In a third aspect of the present application, an electronic device is provided, comprising a processor, a memory, a user interface, and a network interface, the memory being configured to store instructions, the user interface and the network interface being configured to communicate with other devices, and the processor being configured to execute the instructions stored in the memory to enable the electronic device to perform the method of any one of the above aspects.

[0016] In a fourth aspect of the present application, a computer-readable storage medium is provided, the computer-readable storage medium storing a computer program, and the computer program being configured to enable a processor to perform the method of any one of the above aspects.

[0017] The one or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages: 1. Construct a visual internal coordinate system based on the visual anchor point and the visual arc surface of the target driver; generate a reference selection box corresponding to the visual target in the head-up display field of view based on the visual internal coordinate system; output the candidate coordinate points corresponding to the reference selection box in the visual internal coordinate system, and obtain the target coordinate points that meet the preset screening condition from the candidate coordinate points; construct a visual display box corresponding to the visual target based on the target coordinate point, and display the driving environment information in the visual display box to the target driver, wherein the driving environment information includes road condition information, pedestrian information and driving route information, so as to realize consistent geometric alignment and time sequence stable display for driving observation conditions in the panoramic large field of view, reduce position drift and cross-frame jitter, reduce multi-target occlusion and information congestion, and improve the readability and decision-making timeliness of key information.

[0018] 2. Confirm the visual anchor point according to the visual pose parameters of the target driver, wherein the visual pose parameters include head pose parameters, eye gaze point distribution parameters and seat position parameters; determine the visual arc surface based on the spatial geometric relationship between the visual anchor point and the vehicle windshield; construct a heterogeneous scale coordinate system in the visual range of the target driver based on the visual anchor point and the visual arc surface; construct a visual internal coordinate system with the visual anchor point as the origin, the principal tangent and the normal of the visual arc surface as the basis vectors, and the heterogeneous scale coordinate system as the scale reference, so as to form a unified calculation coordinate basis consistent with the driving observation conditions, adapt to the differences in posture and curved surface, reduce misplacement and jitter, and provide stable and checkable geometric constraints and scale consistency for reference selection box generation, candidate coordinate point screening and visual display box presentation.

[0019] 3. Label each candidate coordinate point as a person end point and a vehicle end point in the visual internal coordinate system, and pair each person end point with the nearest vehicle end point to generate a person-vehicle end point pair according to the nearest field principle; calculate the Euclidean distance corresponding to the person-vehicle end point pair, and calculate the projection distance of the person-vehicle end point pair along the principal tangent of the visual arc surface; weight the Euclidean distance and the projection distance to obtain the original person-vehicle interval, and perform exponential moving average on the original person-vehicle interval to obtain the smoothed person-vehicle end point interval; perform deduplication statistics on the same type of end points according to the deduplication radius to calculate the number of person end points and the number of vehicle end points in the current frame; sum the number of person end points and the number of vehicle end points to output the number of person-vehicle end points, so as to drive the threshold determination with unified and low-noise distance and number indicators, reduce pairing errors and cross-frame jitter, stabilize the target screening result, and provide a reliable basis for subsequent display box construction and alarm style switching. BRIEF DESCRIPTION OF DRAWINGS

[0020] Figure 1 is a flowchart of a panoramic head-up display method provided by an embodiment of the present application; Figure 2Is a visual anchor and visual arc surface under the construction of a heterogeneous ratio coordinate system schematic diagram provided by the embodiment of the application; Figure 3 Is a reference selection box construction schematic diagram provided by the embodiment of the application; Figure 4 Is a candidate coordinate point schematic diagram provided by the embodiment of the application; Figure 5 Is a target coordinate point schematic diagram provided by the embodiment of the application; Figure 6 Is a module schematic diagram of a panoramic head-up display device provided by the embodiment of the application; Figure 7 Is a structure schematic diagram of an electronic device provided by the embodiment of the application.

[0021] Marked with the following figure: 61, acquisition module; 62, processing module; 701, processor; 702, communication bus; 703, user interface; 704, network interface; 705, memory. DETAILED DESCRIPTION

[0022] In order for those skilled in the art to better understand the technical solutions in the specification, the technical solutions in the specification will be described clearly and completely in the following description of the embodiments of the specification, and obviously, the described embodiments are only part of the embodiments of the application, not all the embodiments.

[0023] The terms used in the following embodiments of the application are only for the purpose of describing the specific embodiments, and are not intended to be a limitation of the application. As used in the specification of the application, the singular expression "one", "a", "said", "the above", "the" and "this" are intended to also include the plural expression, unless there is clear indication to the contrary in the context. It should also be understood that the term "and / or" used in the application means any or all possible combinations of one or more listed items.

[0024] Hereinafter, the terms "first", "second" are only for the purpose of description, and cannot be understood as implying or implying relative importance or implicitly indicating the number of indicated technical features. Therefore, the features defined with "first", "second" can explicitly or implicitly include one or more features, and in the description of the embodiments of the application, unless otherwise specified, the meaning of "multiple" is two or more.

[0025] In order for those skilled in the art to better understand the technical solutions of the application, the application will be further described in conjunction with the drawings.

[0026] Please refer to Figure 1Fig. 1 shows a flow diagram of a panoramic head-up display method according to an embodiment of the present application. The flow diagram mainly includes the following steps: S101-S104.

[0027] In step S101, a visual internal coordinate system is constructed based on the visual anchor point and the visual arc surface of the target driver.

[0028] Specifically, the visual internal coordinate system is established by determining the visual anchor point, calculating the visual arc surface, and defining the coordinate origin, basis vectors, and scale rules. First, the visual anchor point is determined according to the visual pose parameters of the target driver. Then, the visual arc surface consistent with the curvature of the windshield is obtained using the spatial geometric relationship between the visual anchor point and the windshield. The main tangent and normal of the visual arc surface are taken as the basis vectors, and the anisotropic axial scale coefficients and numerical ranges are set by using heterogeneous scaling to complete the construction of the consistent calculation coordinate frame for subsequent target detection, coordinate point output, and display mapping.

[0029] In one possible implementation, step S101 further includes: confirming the visual anchor point according to the visual pose parameters of the target driver, the visual pose parameters including head pose parameters, eye movement fixation point distribution parameters, and seat position parameters; determining the visual arc surface based on the spatial geometric relationship between the visual anchor point and the windshield of the vehicle; constructing an anisotropic coordinate system within the visual range of the target driver based on the visual anchor point and the visual arc surface; and constructing the visual internal coordinate system with the visual anchor point as the origin, the main tangent and normal of the visual arc surface as the basis vectors, and the anisotropic coordinate system as the scale reference.

[0030] Specifically, the head pose parameters, eye movement fixation point distribution parameters, and seat position parameters are obtained from the visual pose parameters of the target driver, and the visual anchor point is confirmed through geometric and statistical joint estimation. Let the position of the eye reference point in the vehicle coordinate system be vector , the translation reference is given by the seat position parameters, and the attitude rotation matrix is given by the head pose parameters , and the unit fixation ray set is given by the eye movement fixation point distribution parameters and the probability weights Under the geometric model constraints of the windshield implicit curved surface or the parameter surface , the intersection of each fixation ray and the windshield is calculated and a weighted robust fusion is performed to obtain the visual anchor point The calculation process is as follows:

[0031]

[0032]

[0033] where, is the eye reference point position vector, the seat reference displacement is determined by the seat position parameter and its absolute position in the vehicle coordinate system is determined by combining the calibration extrinsic parameters, is a three-dimensional rotation matrix composed of the head pose parameters, is a set of unit gaze direction vectors, obtained by sampling and normalizing the eye movement gaze point distribution parameters through a time window, is the distribution probability or confidence weight corresponding to , is the geometric model function of the windshield, which can be obtained from vehicle manufacturing parameters or offline scanning, is the positive scale factor of the intersection of the ray and the curved surface, is a robust loss function used to suppress the influence of abnormal gaze samples on the estimation of visual anchor points, is the actual intersection of each gaze ray and the windshield, is the final visual anchor point position. The head pose parameters are used to give , the value range is a legal set of three-dimensional rotations, the eye movement gaze point distribution parameters are used to give and and satisfy the weight normalization condition, the seat position parameters are used to give the translation reference and vary within the range of seat adjustment allowed by the vehicle.

[0034] After obtaining the visual anchor point, the visual dome is determined based on the spatial geometric relationship between the visual anchor point and the windshield of the vehicle. Take as the center to take a local neighborhood point set N={ } on the windshield curved surface, is the sampling point set of the windshield model in neighborhood, used to fit the local curvature, through second-order surface fitting or principal curvature approximation to obtain the local principal tangent , and normal , and construct a dome sheet which is continuous with the curvature of the glass shape, its quadratic approximation form is:

[0035] where, and are two mutually orthogonal principal tangents at , is the outer normal at , and are the curvature coefficients corresponding to the principal directions, obtained by least squares fitting on the point set N, and is the arc surface local parameter coordinate. The visual arc surface is used to describe the effective display geometry carrier near the driver's gaze, which keeps normal and curvature continuous with the curvature of the windshield.

[0036] An anisometric coordinate system is constructed in the visual range of the target driver, and anisotropic scale mapping is defined according to the different sensitivities of the visual perception to different directions. 、 、 is the orthogonal basis, the scale matrix is defined, the linear mapping from the physical three-dimensional to the anisometric coordinate is constructed, and the anisometric coordinate of any point x is given as follows:

[0037] wherein, is the scale coefficient, which respectively controls the numerical scaling rules along the principal tangent , the principal tangent and the normal , which is used to map the geometric distance to the scale order in visual calculation, and the value is set according to the information density requirements of the system to the lateral, longitudinal and depth directions, which is a positive real number and can be updated by offline calibration or online adaptive estimation, T is the scale mapping matrix from the world coordinate to the visual local basis, is the anisometric coordinate vector.

[0038] The visual internal coordinate system is constructed with the visual anchor point as the origin, the principal tangent and the normal of the visual arc surface as the basis vectors, and the anisometric coordinate system as the scale reference. The basis matrix of the visual internal coordinate system is defined, the origin is , and the scale reference is given by The visual internal coordinate of any point x is defined as follows:

[0039] wherein, is used to give the direction reference and ensure consistency with the visual arc surface, is used as the scale reference for anisotropic scaling, is the coordinate representation in the visual internal coordinate system, which is used for subsequent coordinate point output and screening of the reference selection box and mapping to the head-up display plane. Please refer to Figure 2 , which shows a construction diagram of an anisometric coordinate system under a visual anchor point and a visual arc surface according to an embodiment of the present application.

[0040] In step S102, a reference selection box corresponding to a visual target in the head-up display field of view is generated based on the visual internal coordinate system. ​​

[0041] Specifically, the detected visual target is normalized into a computable reference bounding box using an internal visual coordinate system. The positioning result and the set of external points of the visual target are first received in the internal visual coordinate system, and an initial boundary circumscribing the target is generated according to the base vector and scale reference of the visual arc surface, to give the geometric parameters and position parameters of the reference bounding box. Then, an identification, a time stamp, and an association relationship with the visual target are assigned to each reference bounding box for subsequent coordinate point output and screening. The reference bounding box only serves as a geometric carrier inside the algorithm and does not contain a presentation style. It is refreshed with the frame and keeps consistent with the coordinate anchor of the visual target to ensure the comparability and time sequence continuity of the subsequent candidate coordinate points.

[0042] In a possible implementation, the step S102 further includes: acquiring a visual target in the head-up display field of view, the visual target including a movable visual target and an immovable visual target; generating an equal-size bounding box corresponding to the movable visual target based on the internal visual coordinate system; and generating an area-type bounding box corresponding to the immovable visual target based on the internal visual coordinate system.

[0043] Specifically, the visual target is acquired in the head-up display field of view where the camera perception and the head-up display projection region overlap. The in-frame labeling and inter-frame association are completed through the target positioning result and the time stamp aligned with the internal visual coordinate system. The spatial position, the set of external key points, and the motion state of each target are marked as the visual target attributes. The targets with continuous speed or trajectory are marked as movable visual targets, and the targets without significant motion are marked as immovable visual targets. The movable and immovable attributes are jointly judged by the threshold and the trajectory fitting to ensure the stability of the attributes in a short time window, so as to avoid frequent switching of the display box form.

[0044] When generating the equal-size bounding box for the movable visual target, the main direction and the length and width parameters of the target are calculated from the set of external key points of the target in the internal visual coordinate system. The main direction is constrained according to the base vector and the scale reference of the visual arc surface, so that the long side of the equal-size bounding box is consistent with the main direction of the target, and the short side is perpendicular to the main direction. A front and back buffer distance consistent with the scale reference is introduced in the depth direction to enhance the readability of the box body when the distance changes. The geometric center of the equal-size bounding box is aligned with the dynamic centroid of the target, and the update of the box body follows the strategy of "detection priority, smooth following", that is, when the positioning of the new frame target is higher than the threshold, the box body is updated with the new positioning, otherwise the box body is maintained based on the last frame box body and the speed extrapolation, to ensure that the box body does not jump under the condition of short-time occlusion or jitter.

[0045] When generating the area selection box for the immovable visual target, a minimum covering rectangle or regular polygon is constructed with the target's outer key point set in the visual internal coordinate system, and the boundary normal is kept consistent with the visual arc surface normal. In the horizontal and vertical directions, pixel density mapping consistent with the scale reference is adopted to ensure the stable display ratio of large area targets at different depths. For immovable visual targets with regular structures such as stop lines, parking spaces, or road signs, shape templates are preferred to constrain the area selection box, and the box edges are aligned along the template main axis to obtain regular layout anchors for subsequent superimposed text and icons.

[0046] When the overlap between the equal-size selection box and the area selection box appears in the same frame, hierarchical clipping and boundary contraction are performed according to the movable priority principle to ensure that the movable visual target is observed first in the overlapping area, and the clipped area selection box is processed with transparency enhancement and stroke thickening in non-critical areas to maintain its recognizability. The life cycle of the reference selection box and the association with the visual target are one-to-one, and the identification is initialized and allocated when the target appears, and the identification is recycled when the target is lost or judged as noise. The geometric parameters, position parameters, and time parameters are consistently updated in the visual internal coordinate system throughout the process. Please refer to Figure 3 , which shows a reference selection box construction diagram provided by an embodiment of the present application, Figure 3 , where the left white line represents the output equal-size selection box, and the right white line represents the output area selection box.

[0047] In step S103, the corresponding candidate coordinate points of the output reference selection box in the visual internal coordinate system are output, and the target coordinate points satisfying the preset screening conditions are obtained from the candidate coordinate points.

[0048] Specifically, the end points and midpoints are extracted according to the geometric structure of the reference selection box, and their coordinates in the visual internal coordinate system are calculated, which are summarized as candidate coordinate points. Quality and constraint verification is performed on the candidate coordinate points, including continuous traceability, geometric consistency, confidence threshold, human-vehicle endpoint interval threshold, endpoint number threshold, and motion speed threshold. The candidate coordinate points that do not meet the threshold or are judged as road endpoints are excluded, and the candidate coordinate points that meet the conditions are de-duplicated, time-series smoothed, and frame-interrelated. The target coordinate points that are stable and can be used for subsequent display and interaction are retained.

[0049] In a possible implementation, step S103 further includes: when the output is the equal-body selection box, extracting four edge endpoints of the equal-body selection box, and obtaining a first set of coordinate points corresponding to the four edge endpoints in the visual-in coordinate system; taking the first set of coordinate points as candidate coordinate points corresponding to the equal-body selection box; when the output is the area selection box, extracting four edge midpoints of the area selection box, and obtaining a second set of coordinate points corresponding to the four edge midpoints in the visual-in coordinate system; and taking the second set of coordinate points as candidate coordinate points corresponding to the equal-body selection box.

[0050] Specifically, when the output is the equal-body selection box, the endpoints of the four edges are read in the geometric description of the selection box, the vertex order is fixed according to the frame timestamp to avoid left-right flipping, the four endpoints are converted from the detection coordinate expression to the visual-in coordinate system expression, and the first set of coordinate points is obtained. When converting, the origin, base vector and scale reference of the visual-in coordinate system are used to map the endpoint position once, and the source edge identifier and confidence of each endpoint are recorded. The first set of coordinate points is de-duplicated and distortion-corrected, and if there are missing points or repeated points caused by edge clipping, the endpoint positions of the previous frame are used for time series interpolation to fill in the missing points. The first set of coordinate points after the above processing is directly used as the candidate coordinate points corresponding to the equal-body selection box, and enters the subsequent threshold judgment and screening process.

[0051] When the output is the area selection box, the midpoints of the four edges are calculated from the geometric equations of the four edges of the selection box, the order consistent with the edge normal is marked, and the four midpoints are converted to the visual-in coordinate system to obtain the second set of coordinate points. In the conversion, the origin, base vector and scale reference of the visual-in coordinate system are also used, and the edge index of the midpoint is retained for subsequent consistency check with the template or scene constraint. The second set of coordinate points is boundary-clipped and time-series-smoothed, and if any edge is missing due to occlusion or out-of-view, the available midpoints of the adjacent two frames are linearly extrapolated to fill in the missing points. The second set of coordinate points after processing is used as the candidate coordinate points corresponding to the area selection box, and enters the same screening and association process as the equal-body selection box. Please refer to Figure 4 which shows a candidate coordinate point diagram provided in the present application. Figure 4 In the figure, the red points are the candidate coordinate points.

[0052] In a possible implementation, step S103 further includes: calculating the interval between the human-vehicle end points and the number of human-vehicle end points based on the candidate coordinate points, specifically including: marking the candidate coordinate points as human end points and vehicle end points in the visual internal coordinate system according to categories, and pairing each human end point with the nearest vehicle end point according to the nearest field principle to generate human-vehicle end point pairs; calculating the Euclidean distance corresponding to the human-vehicle end point pairs, and calculating the projection distance of the human-vehicle end point pairs along the main tangent of the visual surface; weighting the Euclidean distance and the projection distance to obtain the original human-vehicle interval according to a weight; performing exponential sliding average on the original human-vehicle interval according to frames to obtain the human-vehicle end point interval after smoothing processing; performing the de-duplication statistics according to the de-duplication radius for the same type of end points to calculate the number of human end points and the number of vehicle end points in the current frame; performing summation calculation based on the number of human end points and the number of vehicle end points to output the number of human-vehicle end points; judging whether the interval between the human-vehicle end points is less than a first preset threshold; when it is confirmed that the interval between the human-vehicle end points is less than the first preset threshold, the candidate coordinate points are not output; judging whether the interval between the human-vehicle end points is greater than a second preset threshold; when it is confirmed that the interval between the human-vehicle end points is greater than the second preset threshold, the road end points in the candidate coordinate points are removed, and the remaining coordinate points are output as target coordinate points, the first preset threshold is less than the second preset threshold; judging whether the number of human-vehicle end points is greater than a preset number; when it is confirmed that the number of human-vehicle end points is greater than the preset number, the end points with a motion speed greater than a preset speed in the candidate coordinate points are output as target coordinate points.

[0053] Specifically, after the candidate coordinate points are determined, the human end points and the vehicle end points are first marked in the visual internal coordinate system, the interval between the human-vehicle end points is calculated, and the number of human-vehicle end points is counted in the same time window. The interval between the human-vehicle end points is calculated by using the Euclidean distance between the end point pairs or the projection distance along the main tangent of the visual surface, to ensure consistency across frames; the number of human-vehicle end points is counted by de-duplication of the effective end points in the current frame, and the average value in the sliding window across frames is retained for robust determination. The calculation methods of the interval between the human-vehicle end points and the number of human-vehicle end points are as follows: The candidate coordinate points are divided into a human end point set and a vehicle end point set in the visual internal coordinate system, the human end point set is denoted as , and the vehicle end point set is denoted as , both of which are two-dimensional or three-dimensional coordinate vectors in the visual internal coordinate system. The main tangent of the visual surface is taken as , the normal is taken as , and the scale reference is taken as , to keep the measurement consistent across frames. To obtain the interval between the human-vehicle end points of adjacent different types, a human-vehicle end point pair set is constructed based on the nearest field principle, which includes the radius neighborhood constraint and the nearest neighbor constraint, where and , is a basis vector matrix, For the upper limit of the pairing search radius, the radius constraint is not met Not involved in the frame matching. For the radius neighborhood constraint, after the visual-in coordinate system is directionally aligned and scaled, only when the normalized distance from a person endpoint to any vehicle endpoint is not more than the preset upper limit of the search radius , the person endpoint is allowed to participate in pairing; if the normalized distance from all vehicle endpoints to the person endpoint is greater than , it is determined that there is no valid neighborhood, the person endpoint is not paired in the frame and is marked as pending; for the nearest neighbor constraint, among the person endpoint candidate vehicle endpoint set that meets the radius neighborhood constraint, the vehicle endpoint with the smallest normalized distance is selected as the only pairing object; if there are distance ties, the vehicle endpoint with higher confidence and continuous association with the last frame is preferentially selected as the pairing object; if it is still impossible to distinguish, it is stably selected in a fixed order or identification number to avoid cross-frame jitter.

[0054] In the statistics of the number of person and vehicle endpoints, the and are first de-duplicated, and the de-duplication rule is that when the same type of endpoints are in the normalized coordinates and the distance between each other is less than , only the one with higher confidence is retained, and the de-duplicated set is obtained. The number of valid person and vehicle endpoints in the frame is and respectively, and the total number of person and vehicle endpoints is . To make a robust determination, the exponential moving average of the number is calculated on a frame sliding window with a length of , and the time index is set as , then , is the smoothing coefficient, and is initialized. To ensure consistency across frames, the interval between person and vehicle endpoints also uses exponential moving average. For each pairing , its historical identification is established, and it is associated in adjacent frames through nearest neighbor and Hungarian matching, and the historical pairing identification matched is recorded as , then the smoothed interval between person and vehicle endpoints is , is the interval smoothing coefficient, and is initialized. Among them, is the original person-vehicle interval, that is, the interval between person and vehicle endpoints before smoothing.

[0055] The threshold and the determination quantity use the same name and meaning as the claims. The first preset threshold is recorded as , the second preset threshold is recorded as , which satisfies and the units of the two are consistent with ; the preset number is recorded as , used to limit or upper bound; preset speed is denoted as , used for speed filtering when subsequent quantity exceeds limit. upper bound of neighborhood search radius for person-vehicle pair, used to suppress long-distance mispairing; de-duplication radius for same type of endpoints; distance combination weight; sliding window length; and are smoothing coefficients for quantity and interval respectively. principal tangent and normal of visual arc surface, to ensure that the metric direction is consistent with the gaze geometry; used for anisotropic scale normalization, to ensure that distance and quantity statistics in different directions and depths are comparable.

[0056] When the interval between person and vehicle endpoints is less than a first preset threshold, it is determined to be an over-dense state. The over-dense state corresponds to high occlusion or mis-detection risk, and the candidate coordinate points are directly not output, and the target coordinate points of the previous frame are maintained internally to ensure display continuity; if there is no valid target coordinate point in the previous frame, the current frame is empty and does not trigger display update.

[0057] When the interval between person and vehicle endpoints is greater than or equal to the first preset threshold and less than or equal to a second preset threshold, it is determined to be a normal density interval. The normal density interval does not exclude endpoints of the same type, but needs to perform geometric consistency and continuous traceability checks on the candidate coordinate points, remove endpoints with confidence lower than a threshold or cross-frame drift exceeding limit, and retain the candidate coordinate points that pass the checks as target coordinate points.

[0058] When the interval between person and vehicle endpoints is greater than the second preset threshold, it is determined to be an over-sparse state. The over-sparse state is prone to contain background or road interference, and the road endpoints in the candidate coordinate points are removed, and only the remaining coordinate points are output as target coordinate points; if there are no remaining endpoints after removal, the target coordinate points of the current frame are empty, and an over-sparse flag is recorded internally for subsequent threshold adaptive adjustment.

[0059] In the person and vehicle endpoint quantity determination, when the number of person and vehicle endpoints is less than or equal to a preset number, further screening of quantity constraints is not triggered, and the output results of the interval branches described above are directly used to ensure that in the rare target scene, the information is not excessively weakened. When the number of person and vehicle endpoints is greater than the preset number, to avoid information congestion, only the endpoints with a motion speed greater than a preset speed in the candidate coordinate points are output as target coordinate points; if the number of endpoints meeting the speed condition is still too large, the number of endpoints is truncated to not more than the preset number from high to low according to the speed, and the association priority with the previous frame is maintained to reduce display jitter.

[0060] When the interval between the human-vehicle endpoints is exactly equal to the first preset threshold, it is considered to enter the normal density interval, and the geometric consistency and continuous traceability check of the normal density interval is performed. When the interval between the human-vehicle endpoints is exactly equal to the second preset threshold, it is still processed as the normal density interval, and the road surface endpoint elimination is not triggered; only when it is strictly greater than the second preset threshold, the road surface endpoint elimination is performed. When the number of human-vehicle endpoints is exactly equal to the preset number, the number pruning is not triggered, and the current branch output is maintained to avoid boundary jitter. When the number of human-vehicle endpoints is greater than the preset number but there is no endpoint with a speed greater than the preset speed, in order to prevent complete loss of information, the speed of the highest endpoint is selected until the preset number is not exceeded, and the low-confidence target coordinate point is marked for subsequent frame priority update.

[0061] The threshold and the preset number can be updated adaptively according to the scene. When over-dense or over-dilute labels appear in continuous multiple frames, the first preset threshold and the second preset threshold are adjusted slightly. When crowded display appears in continuous multiple frames, the preset speed is increased or the upper limit of the preset number is reduced. All filtered target coordinate points perform deduplication and time smoothing before output, and the endpoint source, category and speed label are retained to ensure one-to-one correspondence with the subsequent visual display box construction process. Please refer to Figure 5 , which shows a target coordinate point diagram provided by an embodiment of the application. Figure 5 In the figure, the red points are the filtered target coordinate points.

[0062] Step S104, based on the target coordinate points, a visual display box corresponding to the visual target is constructed, and driving environment information is displayed to the target driver in the visual display box. The driving environment information includes road condition information, pedestrian information and driving route information.

[0063] Specifically, the target coordinate points are used as geometric anchor points to generate visual display boxes in the head-up display plane, and information superposition and presentation are completed. First, according to the fixed mapping relationship from the visual coordinate system to the head-up display plane, the target coordinate points are converted from visual coordinate expression to display plane coordinate expression, and the mapping maintains consistency with the visual arc surface normal to avoid perspective distortion. The converted target coordinate points are used as boundary control points to directly generate the boundary polygon of the visual display box, and style parameters such as line width, transparency and corner radius are set. The unique identifier and timestamp of the display box are established to maintain a one-to-one binding relationship with the corresponding visual target.

[0064] In the inner and peripheral areas of the visual display box, the layout anchor points of the text and icons derived from the target coordinate points include the upper left corner, the upper right corner, the midpoint of the bottom edge and the outward offset point in the box, which are used to carry specific elements of the driving environment information. The road condition information is presented in the form of text and icons at the outward offset point of the box, and the speed limit, lane attribute, congestion state and distance ahead are displayed in priority; the pedestrian information is presented in the form of small icons and distance readings at the top or side of the box, and is switched to a high-contrast style when the target is close; the driving route information is superimposed along the driving direction of the vehicle in the form of a transparent path band and a steering arrow, and the path band and the edge of the display box maintain a safe distance to avoid blocking the key target. In a multi-target concurrent scene, the display level and priority of the visual display box are assigned, the priority is given by the distance from the ego vehicle, the relative speed and the risk level, and the level is used to determine the occlusion relationship and the transparency adjustment. When the visual display boxes occlude each other, the strategy of rendering the box with higher priority first, automatically shrinking the box with lower priority and semi-transparent outlining is adopted. To ensure the stability of perception, the geometric parameters and position parameters of the display box are exponentially smoothed in consecutive frames, and the minimum visible time and the maximum disconnection buffer time are set. When the target is short-time disconnected and reappears within the buffer time, the historical display box is used to reduce the jump.

[0065] In the state triggering and alarm interaction, when the distance between the human-vehicle endpoints obtained based on the foregoing screening is less than a first preset threshold, the visual display box switches to a high warning style, including thickening the lines, improving the color and slightly flashing, and a short avoidance prompt is popped up at the anchor point outside the box; when the distance between the human-vehicle endpoints is in a normal density interval, the standard style and the normal information density are maintained; when it is in a sparse state and the road endpoint has been excluded, only the core readings are kept to reduce visual noise. The interaction area and the hit determination are updated together with the display box, and the click or focus event is converted into a query or navigation detail request in the vehicle-mounted system without changing the rendering process of this step.

[0066] In the life cycle management, the visual display box is initialized with the creation of the target, refreshed while the target is continuously tracked, and recycled when the target disappears stably or is determined as noise, and the final state and readings are written into the log before recycling for review. During the whole process, the mapping, rendering and information superposition are frame-synchronized, the target coordinate point is the only driving source, the visual display box is the only presentation carrier, and the driving environment information is the only display content, forming a stable closed loop.

[0067] Please refer to Figure 6 which shows a module schematic diagram of a panoramic head-up display device provided by an embodiment of the present application, the device comprising an acquisition module 61 and a processing module 62, wherein, The acquisition module 61 is configured to construct a visual internal coordinate system based on the visual anchor point and the visual arc surface of the target driver, and generate a reference selection box corresponding to a visual target in a head-up display field of view based on the visual internal coordinate system.

[0068] The processing module 62 is configured to output a candidate coordinate point corresponding to the reference selection box in the visual internal coordinate system, and obtain a target coordinate point meeting a preset screening condition from the candidate coordinate point; construct a visual display box corresponding to the visual target based on the target coordinate point, and display driving environment information including road condition information, pedestrian information and driving route information to the target driver in the visual display box.

[0069] In a possible implementation, the acquisition module 61 is configured to construct the visual internal coordinate system based on the visual anchor point and the visual arc surface of the target driver, and specifically includes: confirming the visual anchor point according to visual pose parameters of the target driver, the visual pose parameters including head pose parameters, eye movement fixation point distribution parameters and seat position parameters; determining the visual arc surface based on a spatial geometric relationship between the visual anchor point and a windshield of the vehicle; constructing a heterogeneous coordinate system in a visual range of the target driver based on the visual anchor point and the visual arc surface; and constructing the visual internal coordinate system with the visual anchor point as an origin, with a principal tangent and a normal of the visual arc surface as base vectors, and with the heterogeneous coordinate system as a scale reference.

[0070] In a possible implementation, the acquisition module 61 is configured to generate the reference selection box corresponding to the visual target in the head-up display field of view based on the visual internal coordinate system, and specifically includes: acquiring the visual target in the head-up display field of view, the visual target including a movable visual target and an immovable visual target; generating a full-body selection box corresponding to the movable visual target based on the visual internal coordinate system; and generating an area selection box corresponding to the immovable visual target based on the visual internal coordinate system.

[0071] In a possible implementation, the processing module 62 is configured to output the candidate coordinate point corresponding to the reference selection box in the visual internal coordinate system when the output is the full-body selection box, and specifically includes: extracting four edge endpoints of the full-body selection box, and obtaining a first coordinate point set corresponding to the four edge endpoints in the visual internal coordinate system; and taking the first coordinate point set as the candidate coordinate point corresponding to the full-body selection box.

[0072] In a possible implementation, the processing module 62 is configured to output the candidate coordinate point corresponding to the reference selection box in the visual internal coordinate system when the output is the area selection box, and specifically includes: extracting four edge midpoints of the area selection box, and obtaining a second coordinate point set corresponding to the four edge midpoints in the visual internal coordinate system; and taking the second coordinate point set as the candidate coordinate point corresponding to the area selection box.

[0073] In a possible implementation, the processing module 62 is configured to obtain a target coordinate point from the candidate coordinate points that satisfy a preset screening condition, specifically including: calculating the interval between the human-vehicle end points and the number of human-vehicle end points based on the candidate coordinate points; determining whether the interval between the human-vehicle end points is less than a first preset threshold; when it is confirmed that the interval between the human-vehicle end points is less than the first preset threshold, the candidate coordinate point is not output; determining whether the interval between the human-vehicle end points is greater than a second preset threshold; when it is confirmed that the interval between the human-vehicle end points is greater than the second preset threshold, the road end points in the candidate coordinate point are removed, and the remaining coordinate points are output as the target coordinate points, the first preset threshold is less than the second preset threshold; determining whether the number of human-vehicle end points is greater than a preset number; when it is confirmed that the number of human-vehicle end points is greater than the preset number, the end points with a motion speed greater than a preset speed in the candidate coordinate are output as the target coordinate points.

[0074] In a possible implementation, the processing module 62 is configured to calculate the interval between the human-vehicle end points and the number of human-vehicle end points based on the candidate coordinate points, specifically including: marking the candidate coordinate points as human end points and vehicle end points in the visual internal coordinate system according to categories, and pairing each human end point with the nearest vehicle end point according to the nearest field principle to generate human-vehicle end point pairs; calculating the Euclidean distance corresponding to the human-vehicle end point pairs, and calculating the projection distance of the human-vehicle end point pairs along the main tangent of the visual curved surface; weighting the Euclidean distance and the projection distance to obtain the original human-vehicle interval according to a weight, and performing exponential moving average on the original human-vehicle interval according to frames to obtain the interval between the human-vehicle end points after smoothing processing; performing the de-duplication statistics on the same end points according to the de-duplication radius to calculate the number of human end points and the number of vehicle end points in the current frame; and performing summation calculation based on the number of human end points and the number of vehicle end points to output the number of human-vehicle end points.

[0075] It should be noted that the apparatus provided in the above embodiments is only used to illustrate the division of the above functional modules, and in actual application, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process is detailed in the method embodiments, which will not be repeated here.

[0076] The present application also provides an electronic device. Referring to Figure 7 , Figure 7 is a structural schematic diagram of an electronic device provided by an embodiment of the present application. The electronic device can include at least one processor 701, at least one communication bus 702, a user interface 703, at least one network interface 704, and a memory 705.

[0077] The communication bus 702 is configured to realize the connection and communication between the components.

[0078] The user interface 703 can include a display, a camera, and optionally a standard wired interface and a wireless interface.

[0079] The network interface 704 can optionally include a standard wired interface and a wireless interface (e.g., a WI-FI interface).

[0080] The processor 701 can include one or more processing cores. The processor 701 connects various parts of the server through various interfaces and lines, and performs various functions of the server and processes data by running or executing instructions, programs, code sets or instruction sets stored in the memory 705, and calling data stored in the memory 705. Optionally, the processor 701 can be implemented in at least one of a hardware form of a digital signal processing (DSP), a field-programmable gate array (FPGA), and a programmable logic array (PLA). The processor 701 can be integrated with a combination of one or more of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. The CPU is mainly used to process an operating system, a user interface, and an application program. The GPU is used to render and draw the content to be displayed on the display. The modem is used to process wireless communication. It can be understood that the above-mentioned modem can also not be integrated into the processor 701, but can be implemented by a separate chip.

[0081] The memory 705 can include a random access memory (RAM) and a read-only memory (ROM). Optionally, the memory 705 includes a non-transitory computer-readable storage medium. The memory 705 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 705 can include a program storage area and a data storage area. The program storage area can store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playing function, an image playing function, etc.), instructions for implementing the above-mentioned various method embodiments, etc. The data storage area can store data involved in the above-mentioned various method embodiments, etc. The memory 705 can optionally be at least one storage device located away from the above-mentioned processor 701. For reference Figure 7The memory 705, as a computer storage medium, can include an operating system, a network communication module, a user interface module, and a panoramic head-up display application program.

[0082] In Figure 7 In the electronic device shown, the user interface 703 is mainly used to provide an interface for user input, and obtain data input by the user; and the processor 701 can be used to call the panoramic head-up display application program stored in the memory 705, and when executed by one or more processors 701, make the electronic device execute the method described in one or more of the above embodiments. It should be noted that, for the foregoing method embodiments, in order to simply describe, they are all described as a combination of a series of actions, but those skilled in the art should know that the present application is not limited to the order of the actions described, because according to the present application, certain steps can be performed in other order or simultaneously. Secondly, those skilled in the art should know that the embodiments described in the specification all belong to preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0083] The present application also provides a computer readable storage medium, which stores instructions. When executed by one or more processors, the electronic device executes the method described in one or more of the above embodiments.

[0084] In the above embodiments, the description of each embodiment has its own focus, and the parts not described in detail in a certain embodiment can be referred to the related description of other embodiments.

[0085] In the several embodiments provided by the present application, it should be understood that the disclosed device can be implemented in other ways. For example, the device embodiments described above are only schematic. The division of the units is only a logical function division. There can be another division manner for actual implementation, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections between different parts can be indirect couplings or communication connections through some interfaces, devices or units, and can be electrical or other forms.

[0086] The units described as separate components can or can not be physically separate, and the components displayed as units can or can not be physical units, i.e. can be located in one place, or can be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0087] In addition, each of the functional units in the various embodiments of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0088] If the integrated unit is realized in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable memory. Based on such an understanding, the technical solutions of the present application, essentially or in the part that contributes to the prior art, or all or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a memory and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the various embodiments of the method of the present application. The aforementioned memory includes various media that can store program codes, such as a U disk, a mobile hard disk, a magnetic disk or an optical disk.

[0089] The above is only an exemplary embodiment of the present application, and cannot limit the scope of the present application. That is, any equivalent changes and modifications made in accordance with the teachings of the present application are still within the scope of the present application.

[0090] The present application is intended to cover any variations, uses, or adaptive changes of the present application that follow the general principles of the present application and include common knowledge or conventional technical means in the technical field of the present application that are not described in the present application.

Claims

1. A panoramic head-up display method, characterized in that, The method includes: A visual internal coordinate system is constructed based on the visual anchor point and visual arc surface of the target driver; Based on the aforementioned visual coordinate system, a reference selection box corresponding to the visual target in the head-up display field of view is generated; Output the candidate coordinate points corresponding to the reference selection box in the visual coordinate system, and obtain the target coordinate points that meet the preset filtering conditions from the candidate coordinate points; Based on the target coordinates, a visual display frame corresponding to the visual target is constructed, and driving environment information is displayed to the target driver within the visual display frame. The driving environment information includes road condition information, pedestrian information, and driving route information.

2. The method according to claim 1, characterized in that, The construction of an internal visual coordinate system based on the target driver's visual anchor point and visual arc surface specifically includes: The visual anchor point is determined based on the visual pose parameters of the target driver, including head pose parameters, eye fixation point distribution parameters, and seat position parameters. The visual curved surface is determined based on the spatial geometric relationship between the visual anchor point and the vehicle windshield. Based on the visual anchor point and the visual arc surface, a coordinate system with different scales is constructed within the visual range of the target driver. The visual internal coordinate system is constructed with the visual anchor point as the origin, the principal tangent and normal of the visual arc surface as the basis vectors, and the non-proportional coordinate system as the scale reference.

3. The method according to claim 2, characterized in that, The step of generating a reference selection box corresponding to the visual target in the head-up display field of view based on the visual coordinate system specifically includes: The visual target in the head-up display field of view is acquired, and the visual target includes movable visual targets and immovable visual targets; Based on the visual coordinate system, generate a life-size selection box corresponding to the movable visual target; Based on the aforementioned visual coordinate system, an area-based selection box is generated corresponding to the immovable visual target.

4. The method according to claim 3, characterized in that, When the output is the body-size selection box, the output of the candidate coordinate points corresponding to the reference selection box in the visual coordinate system specifically includes: Extract the four endpoints of the body-size selection box and obtain the first set of coordinate points corresponding to the four endpoints in the visual coordinate system; The first set of coordinate points is used as the candidate coordinate points corresponding to the body-size selection box.

5. The method according to claim 3, characterized in that, When the output is the area-based selection box, the output of the candidate coordinate points corresponding to the reference selection box in the visual coordinate system specifically includes: Extract the midpoints of the four sides of the area selection box, and obtain the set of second coordinate points corresponding to the midpoints of the four sides in the visual coordinate system; The second set of coordinate points is used as the candidate coordinate points corresponding to the body-size selection box.

6. The method according to claim 1, characterized in that, The step of obtaining target coordinate points that meet preset filtering conditions from the candidate coordinate points specifically includes: Calculate the distance between the human and vehicle endpoints and the number of human and vehicle endpoints based on the candidate coordinate points; Determine whether the distance between the human and vehicle endpoints is less than a first preset threshold; When it is confirmed that the interval between the endpoints of the person and the vehicle is less than the first preset threshold, the candidate coordinate point will not be output. Determine whether the distance between the human and vehicle endpoints is greater than a second preset threshold; When it is confirmed that the distance between the endpoints of the vehicle and the pedestrian is greater than the second preset threshold, the road endpoints in the candidate coordinate points are removed, and the remaining coordinate points are output as the target coordinate points. The first preset threshold is less than the second preset threshold. Determine whether the number of the human and vehicle endpoints is greater than a preset number; When it is confirmed that the number of endpoints of the vehicle and the person is greater than the preset number, the endpoints with a movement speed greater than the preset speed among the candidate coordinates are output as the target coordinate points.

7. The method according to claim 6, characterized in that, The calculation of the distance between pedestrian and vehicle endpoints and the number of pedestrian and vehicle endpoints based on the candidate coordinate points specifically includes: In the visual coordinate system, the candidate coordinate points are labeled as human endpoints and vehicle endpoints according to categories, and each human endpoint is paired with the nearest vehicle endpoint according to the proximity principle to generate human-vehicle endpoint pairs. Calculate the Euclidean distance corresponding to the pairing of the human and vehicle endpoints, and calculate the projected distance of the pairing of the human and vehicle endpoints along the principal tangent of the visual arc surface; The Euclidean distance and the projected distance are weighted and fused to obtain the original human-vehicle interval, and the original human-vehicle interval is subjected to an exponential moving average frame by frame to obtain the smoothed human-vehicle endpoint interval. For endpoints of the same type, perform deduplication statistics based on the deduplication radius to calculate the number of human endpoints and vehicle endpoints in the current frame; The summation of the number of human endpoints and the number of vehicle endpoints is used to output the total number of human and vehicle endpoints.

8. A panoramic head-up display device, characterized in that, The device includes an acquisition module and a processing module, wherein, The acquisition module is used to construct a visual coordinate system based on the visual anchor point and visual arc surface of the target driver; and to generate a reference selection box corresponding to the visual target in the head-up display field of view based on the visual coordinate system. The processing module is used to output the candidate coordinate points corresponding to the reference selection box in the visual coordinate system, and obtain the target coordinate points that meet the preset filtering conditions from the candidate coordinate points; construct the visual display box corresponding to the visual target based on the target coordinate points, and display driving environment information to the target driver in the visual display box, the driving environment information including road condition information, pedestrian information and driving route information.

9. An electronic device, characterized in that, The device includes a processor, a communication bus, a user interface, a network interface, and a memory. The memory is used to store instructions. The user interface and the network interface are used to communicate with other devices. The processor is used to execute the instructions stored in the memory to cause the electronic device to perform the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed, perform the method as described in any one of claims 1 to 7.

Images