Display method, device, electronic device and storage medium

Abstract

The present application discloses a display method, device, electronic device and storage medium, and relates to the computer field, in particular to the artificial intelligence fields such as autonomous driving, vehicle networking, smart cockpit, computer vision, etc. The specific implementation scheme is: obtaining an image to be processed; determining the position of the drawn lane lines and the position of the actual lane lines in the image to be processed by identifying the lane lines in the image to be processed, wherein the drawn lane lines are the lane lines displayed by the augmented reality head-up display system AR-HUD; determining the real scene fit of the AR-HUD according to the position of the drawn lane lines and the position of the actual lane lines. The method can calculate the real scene fit of the AR-HUD according to the position of the drawn lane lines and the position of the actual lane lines in the image to be processed, so that the degree of fit between the information displayed by the AR-HUD and the real scene can be determined according to the real scene fit of the AR-HUD.

Description

Technical Field

[0001] The present application relates to the field of computer technology, in particular to artificial intelligence fields such as autonomous driving, vehicle networking, smart cockpits, computer vision, and more specifically to a display method, device, electronic device, and storage medium. Background Art

[0002] AR-HUD (Augmented Reality-Head Up Display) integrates AR technology into HUD, overlaying digital images on the real world, so that the information projected by HUD is integrated with the real driving environment. AR-HUD can display AR animations that match the actual road scene in real time in the driver's visible area.

[0003] Since AR-HUD is fitted with the real scene and can only be identified by the naked eye, it is difficult to judge the degree of fit between AR-HUD and the real scene. Summary of the invention

[0004] The present application provides a display method, device, electronic device and storage medium. The specific scheme is as follows:

[0005] According to one aspect of the present application, a display method is provided, comprising:

[0006] Get the image to be processed;

[0007] Determining the position of the drawn lane line and the position of the actual lane line in the image to be processed by identifying the lane line in the image to be processed, wherein the drawn lane line is the lane line displayed by the AR-HUD;

[0008] The degree of fit of the AR-HUD to the real scene is determined according to the position of the drawn lane line and the position of the actual lane line.

[0009] According to another aspect of the present application, a display device is provided, including:

[0010] An acquisition module, used for acquiring an image to be processed;

[0011] A first determination module is used to determine the position of the drawn lane line and the position of the actual lane line in the image to be processed by identifying the lane line in the image to be processed, wherein the drawn lane line is the lane line displayed by the AR-HUD;

[0012] The second determination module is used to determine the real scene fit of the AR-HUD according to the position of the drawn lane line and the position of the actual lane line.

[0013] According to another aspect of the present application, there is provided an electronic device, including:

[0014] at least one processor; and

[0015] a memory communicatively connected to the at least one processor; wherein,

[0016] The memory stores instructions that can be executed by the at least one processor. The instructions are executed by the at least one processor so that the at least one processor can perform the method described in the above embodiment.

[0017] According to another aspect of the present application, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute the method according to the above embodiment.

[0018] According to another aspect of the present application, a computer program product is provided, including a computer program, wherein the computer program implements the steps of the method described in the above embodiment when executed by a processor.

[0019] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present application.

[0021] Figure 1 A schematic diagram of a flow chart of a display method provided in an embodiment of the present application;

[0022] Figure 2 A schematic diagram of an image to be processed provided in an embodiment of the present application;

[0023] Figure 3 A schematic diagram of a flow chart of a display method provided in another embodiment of the present application;

[0024] Figure 4 A schematic diagram of a flow chart of a display method provided in another embodiment of the present application;

[0025] Figure 5 A schematic diagram of the structure of a display device provided in one embodiment of the present application;

[0026] Figure 6 It is a block diagram of an electronic device used to implement the display method of an embodiment of the present application. DETAILED DESCRIPTION

[0027] The following is a description of exemplary embodiments of the present application in conjunction with the accompanying drawings, including various details of the embodiments of the present application to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present application. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0028] The display method, device, electronic device and storage medium of the embodiments of the present application are described below with reference to the accompanying drawings.

[0029] Artificial intelligence is a discipline that studies the use of computers to simulate certain human thought processes and intelligent behaviors (such as learning, reasoning, thinking, planning, etc.). It includes both hardware-level technical fields and software-level technologies. Artificial intelligence hardware technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, and big data processing; artificial intelligence software technologies include computer vision technology, speech recognition technology, natural language processing technology, as well as deep learning, big data processing technology, knowledge graph technology, and other major directions.

[0030] The concept of Internet of Vehicles originates from the Internet of Things, namely the Internet of Vehicles. It takes moving vehicles as information perception objects and uses the new generation of information and communication technology to realize network connection between vehicles and X (i.e. vehicles, people, roads, and service platforms), improve the overall intelligent driving level of vehicles, and provide users with safe, comfortable, intelligent, and efficient driving experience and transportation services. At the same time, it improves traffic operation efficiency and enhances the intelligence level of social transportation services.

[0031] A smart cockpit refers to the transformation of the vehicle's interior seating space to make the driving and riding experience more comfortable and intelligent.

[0032] Computer vision is a science that studies how to make machines "see". It refers to using cameras and computers to replace human eyes to identify, track and measure targets, and further perform graphic processing so that the computer processes the images into images that are more suitable for human eye observation or transmission to instruments for detection.

[0033] Figure 1 A schematic diagram of a flow chart of a display method provided in accordance with an embodiment of the present application.

[0034] The display method of the embodiment of the present application can be executed by the display device of the embodiment of the present application. The device can be configured in an electronic device to calculate the real-scene fit of the AR-HUD based on the position of the lane line drawn in the image to be processed and the position of the actual lane line, so that the degree of fit between the information displayed by the AR-HUD and the real scene can be determined based on the real-scene fit of the AR-HUD.

[0035] Among them, the electronic device can be any device with computing capabilities, such as a personal computer, a mobile terminal, a server, etc. The mobile terminal can be, for example, a vehicle-mounted device, a mobile phone, a tablet computer, a personal digital assistant, a wearable device, and other hardware devices with various operating systems, touch screens and / or display screens.

[0036] like Figure 1 As shown, the display method includes:

[0037] Step 101: Obtain an image to be processed.

[0038] In the present application, the image to be processed may be obtained by photographing the front of the vehicle in a vehicle using AR-HUD.

[0039] For example, the driver in the vehicle wears a head-mounted camera, and while the vehicle is actually on the road, the head-mounted camera captures the image in front of the vehicle to obtain the image to be processed, or while the vehicle is actually on the road, the image in front of the vehicle is collected at a fixed position in the vehicle to obtain the image to be processed.

[0040] For vehicles using AR-HUD, during driving, AR-HUD can project AR navigation information into the driver's visual area to display and integrate it with the real scene, so that the image taken in front of the vehicle can include the real road scene and AR navigation information. Among them, AR navigation information includes drawn lane lines, vehicle speed, driving direction guide arrows, etc.

[0041] In the present application, the image to be processed may be a single captured image, or may be captured from a captured video.

[0042] Step 102 , by identifying the lane lines in the image to be processed, the positions of the drawn lane lines and the actual lane lines in the image to be processed are determined.

[0043] Among them, the drawn lane lines are the lane lines displayed by AR-HUD in the driver's visible area, the drawn lane lines are the lane lines drawn by AR; and the actual lane lines are the real lane lines on the road.

[0044] In this application, the image to be processed can be preprocessed, such as graying and smoothing, to reduce image noise and remove details, improve image recognition accuracy, and perform edge extraction on the processed image to obtain edge pixels with strong gradient changes. After that, the edge pixels can be connected, and the actual lane lines and drawn lane lines in the image to be processed can be determined based on the matching of the connected shape with the shape of the actual lane line and the shape of the drawn lane line. For example, if the shape obtained by connecting the lines is an arrow shape, then the shape obtained by connecting the lines is the drawn lane line.

[0045] Since the drawn lane lines in the image to be processed are displayed by the AR-HUD, when determining the position of the drawn lane lines, the imaging position of the AR-HUD in the image to be processed can also be determined according to the imaging area of ​​the AR-HUD, and the target area of ​​the image to be processed can be determined according to the imaging position of the AR-HUD. For example, Figure 2 The area in the box shown is the target area. After that, the target area can be preprocessed, the target area can be converted into a grayscale image, and the edge of the target area can be extracted to obtain edge pixels in the target area, and the edge pixels are connected. The shape after the connection matches the preset shape is the drawn lane line, so that the position of the drawn lane line can be determined.

[0046] Therefore, by determining the target area in the image to be processed according to the imaging area of ​​the AR-HUD, and determining the position of the drawn lane line from the target area, the accuracy of the determined position of the drawn lane line is improved and the amount of calculation is reduced.

[0047] In the present application, the position of the drawn lane line may include the position of each pixel point on the drawn lane point, and the position of the actual lane line may include the position of each pixel point on the actual lane line.

[0048] Step 103: Determine the degree of fit of the AR-HUD to the real scene according to the position of the drawn lane line and the position of the actual lane line.

[0049] Among them, the real-scene fit of AR-HUD refers to the degree of fit between the drawn lane lines and the real scene, where the real scene refers to the actual lane lines.

[0050] In this application, the distance between the drawn lane line and the actual lane line can be determined according to the position of the drawn lane line and the position of the actual lane line, and then the real scene fit of the AR-HUD can be determined according to the distance. Among them, the smaller the distance between the drawn lane line and the actual lane line, the better the real scene fit of the AR-HUD can be considered.

[0051] Since the real-scene fit of AR-HUD can reflect the degree of fit between the information displayed by AR-HUD and the real scene, for example, it can reflect the accuracy of the road guidance information displayed by AR-HUD. The higher the real-scene fit of AR-HUD, the higher the accuracy of the road guidance information displayed by AR-HUD. Therefore, it can be determined whether AR-HUD meets the acceptance requirements based on the real-scene fit of AR-HUD. If the real-scene fit of AR-HUD does not meet the requirements, AR-HUD can be improved.

[0052] by Figure 2 For example, the accuracy of the road guidance sign d displayed by the AR-HUD can be determined based on the determined real-scene fit of the AR-HUD.

[0053] In the embodiment of the present application, by acquiring the image to be processed and identifying the lane lines in the image to be processed, the position of the drawn lane lines in the image to be processed and the position of the actual lane lines are determined, and then the real scene fit of the AR-HUD is determined according to the position of the drawn lane lines and the position of the actual lane lines. Thus, the real scene fit of the AR-HUD can be calculated according to the position of the drawn lane lines in the image to be processed and the position of the actual lane lines, so that the fit degree of the information displayed by the AR-HUD and the real scene can be determined according to the real scene fit of the AR-HUD.

[0054] Figure 3 A flowchart of a display method provided in another embodiment of the present application.

[0055] like Figure 3 As shown, the display method includes:

[0056] Step 301: Obtain an image to be processed.

[0057] Step 302: By identifying the lane lines in the image to be processed, the positions of the drawn lane lines and the actual lane lines in the image to be processed are determined.

[0058] In the present application, step 301 to step 302 are similar to those described in the above embodiment, so they are not described in detail here.

[0059] Step 303: Acquire multiple pixel point groups from the drawn lane line and the actual lane line, wherein each pixel point group includes a first pixel point on the drawn lane line and a second pixel point on the actual lane line.

[0060] In the present application, the image to be processed can be traversed vertically to determine multiple vertical coordinates, and for each vertical coordinate, the first pixel point on the drawn lane line and the second pixel point on the actual lane line are obtained. Thus, multiple pixel point groups can be obtained from the drawn lane line and the actual lane line, each pixel point group includes the first pixel point on the drawn lane line and the second pixel point on the actual lane line, and the vertical coordinates of the first pixel point and the second pixel point are the same.

[0061] For ease of understanding, the following Figure 2 To explain, Figure 2 Pixel point A1 on the actual lane line a, pixel point B1 on the drawn lane line b, and pixel point C1 on the actual lane line belong to one pixel point group; pixel point A2 on the actual lane line a, pixel point B2 on the drawn lane line b, and pixel point C2 on the actual lane line belong to one pixel point group.

[0062] Step 304 : for each pixel group, determine the sub-real scene fit degree corresponding to each pixel group according to the position of the first pixel and the position of the second pixel.

[0063] For each pixel group, the distance between the first pixel and the second pixel can be determined according to the position of the first pixel and the position of the second pixel, and the sub-real scene fit corresponding to each pixel group can be determined according to the distance between the first pixel and the second pixel.

[0064] In some scenes, there may be multiple lane lines on the road. If multiple actual lane lines are extracted from the image to be processed, when determining the sub-real scene fit corresponding to each pixel point group, the target lane line and the associated lane line of the target lane line can be determined from the multiple actual lane lines according to the positions of the multiple actual lane lines, and the first distance between the drawn lane line and the target lane line is determined according to the position of the first pixel point and the position of the second pixel point on the target lane line, and the second distance between the target lane line and the associated lane line is determined according to the position of the second pixel point on the target lane line and the position of the second pixel point on the associated lane line, and then the sub-real scene fit is determined according to the first distance and the second distance.

[0065] Therefore, when there are multiple actual lane lines, the sub-real scene fit can be determined based on the distance between the drawn lane line and the target lane line, and the distance between the target lane line and the associated lane line, meeting the real scene fit calculation requirements for various scenarios.

[0066] In actual applications, when a vehicle is driving on the road, there is usually a lane line on each side of the vehicle. Based on this, in this application, when determining the target lane line and the associated lane line, the lane lines on the left and right sides of the lane where the vehicle is located can be determined from the multiple actual lane lines according to the positions of the multiple actual lane lines, and the target lane line can be determined from the two actual lane lines according to the road driving rules, and the other actual lane line can be determined as the associated lane line. Among them, the road driving rules include driving on the right and driving on the left. For example, if the road driving rules are to drive on the right, then the lane line on the left of the two actual lane lines can be used as the target lane line, and the other lane line can be used as the associated lane line.

[0067] Alternatively, the actual lane line located in the preset area of ​​the image to be processed may be determined as the target lane line based on the positions of multiple actual lane lines, and then the road driving rules are obtained, and the target direction is determined based on the road driving rules, and the associated lane line of the target lane line is determined from the multiple actual lane lines based on the target direction. The preset area may be the central area of ​​the image to be processed, the target lane line is a lane line of the lane where the vehicle is located, and the size of the preset area may be determined based on the size of the image to be processed, which is not limited in this application; the target direction here may be understood as the orientation of the associated lane line relative to the target lane line.

[0068] for example, Figure 2 In the example, the actual lane line a is the target lane line, and the road driving rule is to drive on the right. It can be determined that the target direction is the right, and an actual lane line b adjacent to the right side of the target lane line can be used as the associated lane line. If the road driving rule is to drive on the left, it can be determined that the target direction is the left, and an actual lane line adjacent to the left side of the target lane line can be used as the associated lane line.

[0069] Therefore, by determining the associated lane line of the target lane line according to the road driving rules, the needs of various driving scenarios are met.

[0070] In the present application, since the vertical coordinates of the first pixel point and the second pixel point are the same, the first distance between the drawn lane line and the target lane line can be determined based on the difference between the horizontal coordinate of the first pixel point and the horizontal coordinate of the second pixel point on the target lane line, and the second distance between the target lane line and the associated lane line can be determined based on the difference between the horizontal coordinate of the second pixel point on the target lane line and the horizontal coordinate of the associated lane line.

[0071] After determining the first distance and the second distance, the ratio between the first distance and the second distance can be used as the real scene fit of the AR-HUD, thereby achieving a measurement of the fit between the AR-HUD and the real scene with high accuracy. Alternatively, the real scene fit of the AR-HUD can also be determined based on the difference between the first distance and the second distance, wherein the greater the difference, the higher the real scene fit.

[0072] for example, Figure 2 In the figure, the actual lane line a is the target lane line, the actual lane line c is the associated lane line of the target lane line, the pixel point A1 on the actual lane line a, the pixel point B1 on the drawn lane line b, and the pixel point C1 on the actual lane line belong to a pixel point group. For the pixel point group, the first distance d1 can be determined according to the difference between the horizontal coordinate of the pixel point A1 and the pixel point B1, and the second distance d2 can be determined according to the difference between the horizontal coordinate of the pixel point A1 and the pixel point C1. The ratio of d1 to d2 can be used as the sub-real scene fit corresponding to the pixel point group.

[0073] Step 305 : determining the real scene fit of the AR-HUD according to the sub-real scene fits corresponding to the plurality of pixel point groups.

[0074] In the present application, the average value of the sub-real scene fit corresponding to multiple pixel point groups can be calculated, and the average value can be determined as the real scene fit of the AR-HUD.

[0075] In an embodiment of the present application, when determining the real scene fit of the AR-HUD according to the position of the drawn lane line and the position of the actual lane line, multiple pixel point groups can be obtained from the drawn lane line and the actual lane line, wherein each pixel point group includes a first pixel point on the drawn lane line and a second pixel point on the actual lane line; for each pixel point group, the sub-real scene fit corresponding to each pixel point group is determined according to the position of the first pixel point and the position of the second pixel point; the real scene fit of the AR-HUD is determined according to the sub-real scene fit corresponding to the multiple pixel point groups. Thus, by determining each sub-real scene fit corresponding to each pixel point group, the real scene fit of the AR-HUD is determined using multiple sub-real scene fits, thereby improving the accuracy of the real scene fit.

[0076] Figure 4 A flowchart of a display method provided in another embodiment of the present application.

[0077] like Figure 4 As shown, the display method includes:

[0078] Step 401: Obtain an image to be processed.

[0079] Step 402 , by identifying the lane lines in the image to be processed, the positions of the drawn lane lines and the actual lane lines in the image to be processed are determined.

[0080] Step 403: Acquire multiple pixel point groups from the drawn lane line and the actual lane line, wherein each pixel point group includes a first pixel point on the drawn lane line and a second pixel point on the actual lane line.

[0081] Step 404 : for each pixel group, determine the sub-real scene fit degree corresponding to each pixel group according to the position of the first pixel and the position of the second pixel.

[0082] In the present application, steps 401 to 404 are similar to those described in the above embodiment, and therefore will not be described again here.

[0083] Step 405: Obtain weights of multiple pixel groups.

[0084] In the present application, the weight of each pixel group may be preset or determined according to the position of the pixels in each pixel group.

[0085] For example, the vertical coordinates of each pixel in each pixel group are the same, and different pixel groups have different corresponding vertical coordinates. Since the pixel group in the image to be processed is closer to the camera that captured the image to be processed, its contribution to the real-scene fit of the AR-HUD is greater. Therefore, the weight corresponding to each pixel group can be obtained according to the size of the vertical coordinate corresponding to each pixel. If a two-dimensional coordinate system is established with the pixel in the lower left corner of the image to be processed as the origin, the smaller the vertical coordinate corresponding to the pixel group, the greater the weight of the pixel group, and the sum of the weights of each pixel group is equal to 1.

[0086] Therefore, by obtaining the weight corresponding to each pixel point group according to the size of the vertical coordinate corresponding to each pixel point group, the calculation accuracy of the real scene fit is improved.

[0087] Step 406 , performing a weighted summation of the sub-real scene fits corresponding to the multiple pixel point groups according to the weights of the multiple pixel point groups, so as to determine the real scene fit of the AR-HUD.

[0088] In this application, the sub-real scene fit degree corresponding to the multiple pixel point groups can be weighted and summed according to the weights of the multiple pixel point groups, and the calculation result can be determined as the real scene fit degree of the AR-HUD. Thus, based on the weights of the multiple pixel point groups, the final real scene fit degree is determined to improve the accuracy of the calculation result.

[0089] In the embodiment of the present application, when determining the real scene fit of the AR-HUD according to the sub-real scene fit corresponding to the multiple pixel point groups, the weights of the multiple pixel point groups can be obtained, and the sub-real scene fit corresponding to the multiple pixel point groups can be weighted summed according to the weights of the multiple pixel point groups to determine the real scene fit of the AR-HUD. Thus, the real scene fit of the AR-HUD is determined based on the weights of the multiple pixel point groups, which improves the accuracy of the calculation of the real scene fit of the AR-HUD.

[0090] In order to implement the above embodiment, the embodiment of the present application further proposes a display device. Figure 5 A schematic diagram of the structure of a display device provided in one embodiment of the present application.

[0091] like Figure 5 As shown, the display device 500 includes:

[0092] An acquisition module 510 is used to acquire an image to be processed;

[0093] A first determination module 520 is used to determine the position of the drawn lane line and the position of the actual lane line in the image to be processed by identifying the lane line in the image to be processed, wherein the drawn lane line is the lane line displayed by the AR-HUD;

[0094] The second determination module 530 is used to determine the real scene fit of the AR-HUD according to the position of the drawn lane line and the position of the actual lane line.

[0095] In a possible implementation of the embodiment of the present application, the second determining module 530 includes:

[0096] An acquisition unit, used to acquire a plurality of pixel point groups from the drawn lane line and the actual lane line, wherein each pixel point group includes a first pixel point on the drawn lane line and a second pixel point on the actual lane line;

[0097] A first determining unit is used to determine, for each pixel point group, a sub-real scene fit degree corresponding to each pixel point group according to a position of the first pixel point and a position of the second pixel point;

[0098] The second determining unit is used to determine the real scene fitting degree of the AR-HUD according to the sub-real scene fitting degrees respectively corresponding to the multiple pixel point groups.

[0099] In a possible implementation of the embodiment of the present application, the number of actual lane lines is multiple, and the first determination unit is used to:

[0100] Determine a target lane line and a lane line associated with the target lane line from a plurality of actual lane lines;

[0101] Determine a first distance between the drawn lane line and the target lane line according to the position of the first pixel point and the position of the second pixel point on the target lane line;

[0102] Determine a second distance between the target lane line and the associated lane line according to the position of the second pixel point on the target lane line and the position of the second pixel point on the associated lane line;

[0103] The degree of fit of the sub-real scene is determined according to the first distance and the second distance.

[0104] In a possible implementation manner of the embodiment of the present application, the first determining unit is configured to:

[0105] According to the positions of multiple actual lane lines, an actual lane line located in a preset area of ​​the image to be processed is determined as a target lane line;

[0106] Get the rules of the road;

[0107] According to the road driving rules, the associated lane line of the target lane line is determined from multiple actual lane lines.

[0108] In a possible implementation manner of the embodiment of the present application, the first determining unit is configured to:

[0109] The sub-real scene fit is determined according to a ratio between the first distance and the second distance.

[0110] In a possible implementation manner of the embodiment of the present application, the acquiring unit is used to:

[0111] Get the weights of multiple pixel groups;

[0112] According to the weights of the multiple pixel groups, the sub-real scene fit corresponding to the multiple pixel groups is weighted and summed to determine the real scene fit of the AR-HUD.

[0113] In a possible implementation manner of the embodiment of the present application, the vertical coordinates of the pixels in each pixel group are the same, and the acquisition unit is used to:

[0114] According to the size of the ordinate corresponding to each pixel group, the weight corresponding to each pixel group is obtained.

[0115] In a possible implementation of the embodiment of the present application, the first determining module 520 is configured to:

[0116] According to the imaging area of ​​the AR-HUD, the target area is determined from the image to be processed;

[0117] Perform edge extraction on the target area to obtain edge pixel points in the target area;

[0118] Determine the location of drawing lane lines based on edge pixels.

[0119] It should be noted that the explanation of the above display method embodiment is also applicable to the display device of this embodiment, so it will not be repeated here.

[0120] In the embodiment of the present application, by acquiring the image to be processed and identifying the lane lines in the image to be processed, the position of the drawn lane lines in the image to be processed and the position of the actual lane lines are determined, and then the real scene fit of the AR-HUD is determined according to the position of the drawn lane lines and the position of the actual lane lines. Thus, the real scene fit of the AR-HUD can be calculated according to the position of the drawn lane lines in the image to be processed and the position of the actual lane lines, so that the fit degree of the information displayed by the AR-HUD and the real scene can be determined according to the real scene fit of the AR-HUD.

[0121] According to an embodiment of the present application, the present application also provides an electronic device, a readable storage medium and a computer program product.

[0122] Figure 6A schematic block diagram of an example electronic device 600 that can be used to implement an embodiment of the present application is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or required herein.

[0123] like Figure 6 As shown, the device 600 includes a computing unit 601, which can perform various appropriate actions and processes according to a computer program stored in a ROM (Read-Only Memory) 602 or a computer program loaded from a storage unit 608 to a RAM (Random Access Memory) 603. In the RAM 603, various programs and data required for the operation of the device 600 can also be stored. The computing unit 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An I / O (Input / Output) interface 605 is also connected to the bus 604.

[0124] A number of components in the device 600 are connected to the I / O interface 605, including: an input unit 606, such as a keyboard, a mouse, etc.; an output unit 607, such as various types of displays, speakers, etc.; a storage unit 608, such as a disk, an optical disk, etc.; and a communication unit 609, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 609 allows the device 600 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0125] The computing unit 601 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a CPU (Central Processing Unit), a GPU (Graphic Processing Units), various dedicated AI (Artificial Intelligence) computing chips, various computing units running machine learning model algorithms, a DSP (Digital Signal Processor), and any appropriate processor, controller, microcontroller, etc. The computing unit 601 performs the various methods and processes described above, such as a display method. For example, in some embodiments, the display method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 608. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 600 via the ROM 602 and / or the communication unit 609. When the computer program is loaded into the RAM 603 and executed by the computing unit 601, one or more steps of the display method described above may be performed. Alternatively, in other embodiments, the computing unit 601 may be configured to execute the display method in any other appropriate manner (eg, by means of firmware).

[0126] Various embodiments of the systems and techniques described above herein may be implemented in digital electronic circuit systems, integrated circuit systems, FPGAs (Field Programmable Gate Arrays), ASICs (Application-Specific Integrated Circuits), ASSPs (Application Specific Standard Products), SOCs (System On Chips), CPLDs (Complex Programmable Logic Devices), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include: being implemented in one or more computer programs that may be executed and / or interpreted on a programmable system including at least one programmable processor that may be a dedicated or general-purpose programmable processor that may receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0127] The program code for implementing the method of the present application can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, implements the functions / operations specified in the flow chart and / or block diagram. The program code can be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0128] In the context of the present application, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any suitable combination of the foregoing. More specific examples of machine-readable storage media may include electrical connections based on one or more lines, portable computer disks, hard disks, RAM, ROM, EPROM (Electrically Programmable Read-Only-Memory) or flash memory, optical fiber, CD-ROM (Compact Disc Read-Only Memory), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0129] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (Cathode-Ray Tube) or LCD (Liquid Crystal Display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0130] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: LAN (Local Area Network), WAN (Wide Area Network), the Internet, and blockchain networks.

[0131] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or a cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services (Virtual Private Server). The server may also be a server of a distributed system, or a server combined with a blockchain.

[0132] According to an embodiment of the present application, the present application also provides a computer program product, which, when an instruction processor in the computer program product is executed, executes the display method proposed in the above embodiment of the present application.

[0133] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this application can be executed in parallel, sequentially or in different orders, as long as the expected results of the technical solution disclosed in this application can be achieved, and this document is not limited here.

[0134] The above specific implementations do not constitute a limitation on the protection scope of this application. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of this application should be included in the protection scope of this application.

Claims

1. A display method, comprising: Get the image to be processed; Determining the position of the drawn lane line and the position of the actual lane line in the image to be processed by identifying the lane line in the image to be processed, wherein the drawn lane line is the lane line displayed by the augmented reality-head-up display system AR-HUD; Acquire multiple pixel point groups from the drawn lane line and the actual lane line, wherein each of the pixel point groups includes a first pixel point on the drawn lane line and a second pixel point on the actual lane line; For each pixel point group, determining a sub-real scene fit degree corresponding to each pixel point group according to a position of the first pixel point and a position of the second pixel point; Determining the real scene fit of the AR-HUD according to the sub-real scene fits corresponding to the multiple pixel point groups respectively; There are multiple actual lane lines, and determining the sub-real scene fit corresponding to each pixel group according to the position of the first pixel point and the position of the second pixel point includes: Determine a target lane line and lane lines associated with the target lane line from a plurality of actual lane lines; Determine a first distance between the drawn lane line and the target lane line according to the position of the first pixel point and the position of the second pixel point on the target lane line; Determine a second distance between the target lane line and the associated lane line according to a position of a second pixel point on the target lane line and a position of a second pixel point on the associated lane line; The sub-real scene fit degree is determined according to the first distance and the second distance.

2. The method of claim 1, wherein: The step of determining a target lane line and a lane line associated with the target lane line from a plurality of actual lane lines includes: According to the positions of the multiple actual lane lines, an actual lane line located in a preset area of ​​the image to be processed is determined as the target lane line; Get the rules of the road; According to the road driving rule, a lane line associated with the target lane line is determined from the multiple actual lane lines.

3. The method of claim 1, wherein: The determining the sub-real scene fit degree according to the first distance and the second distance includes: The sub-reality scene fit degree is determined according to a ratio between the first distance and the second distance.

4. The method of claim 1, wherein: The determining the real scene fit degree of the AR-HUD according to the sub-real scene fit degrees respectively corresponding to the plurality of pixel point groups includes: Obtaining weights of the plurality of pixel groups; According to the weights of the multiple pixel groups, weighted summation is performed on the sub-real scene fits corresponding to the multiple pixel groups to determine the real scene fit of the AR-HUD.

5. The method of claim 4, wherein: The vertical coordinates of the pixels in each pixel group are the same, and obtaining the weights of the plurality of pixel groups includes: According to the size of the ordinate corresponding to each pixel point group, the weight corresponding to each pixel point group is obtained.

6. The method of claim 1, wherein: The step of determining the position of the lane line drawn in the image to be processed by identifying the lane line in the image to be processed includes: According to the imaging area of ​​the AR-HUD, determining a target area from the image to be processed; Performing edge extraction on the target area to obtain edge pixel points in the target area; The position of drawing the lane line is determined according to the edge pixel point.

7. A display device, comprising: An acquisition module, used for acquiring an image to be processed; A first determination module is used to determine the position of the drawn lane line and the position of the actual lane line in the image to be processed by identifying the lane line in the image to be processed, wherein the drawn lane line is the lane line displayed by the AR-HUD; An acquisition unit, configured to acquire a plurality of pixel point groups from the drawn lane line and the actual lane line, wherein each of the pixel point groups includes a first pixel point on the drawn lane line and a second pixel point on the actual lane line; A first determining unit is used to determine, for each pixel point group, a sub-real scene fit degree corresponding to each pixel point group according to a position of the first pixel point and a position of the second pixel point; A second determining unit, configured to determine a degree of fit of the AR-HUD to the real scene according to the degrees of fit of the sub-real scenes respectively corresponding to the plurality of pixel point groups; Wherein, the number of the actual lane lines is multiple, and the first determining unit is used to: Determine a target lane line and lane lines associated with the target lane line from a plurality of actual lane lines; Determine a first distance between the drawn lane line and the target lane line according to the position of the first pixel point and the position of the second pixel point on the target lane line; Determine a second distance between the target lane line and the associated lane line according to a position of a second pixel point on the target lane line and a position of a second pixel point on the associated lane line; The sub-real scene fit degree is determined according to the first distance and the second distance.

8. The device according to claim 7, wherein: The first determining unit is configured to: According to the positions of the multiple actual lane lines, an actual lane line located in a preset area of ​​the image to be processed is determined as the target lane line; Get the rules of the road; According to the road driving rule, a lane line associated with the target lane line is determined from the multiple actual lane lines.

9. The device according to claim 7, wherein: The first determining unit is configured to: The sub-reality scene fit degree is determined according to a ratio between the first distance and the second distance.

10. The device according to claim 7, wherein: The acquisition unit is used to: Obtaining weights of the plurality of pixel groups; According to the weights of the multiple pixel groups, weighted summation is performed on the sub-real scene fits corresponding to the multiple pixel groups to determine the real scene fit of the AR-HUD.

11. The device of claim 10, wherein: The vertical coordinates of the pixels in each pixel group are the same, and the acquisition unit is used to: According to the size of the ordinate corresponding to each pixel point group, the weight corresponding to each pixel point group is obtained.

12. The device of claim 7, wherein: The first determining module is used to: According to the imaging area of ​​the AR-HUD, determining a target area from the image to be processed; Performing edge extraction on the target area to obtain edge pixel points in the target area; The position of drawing the lane line is determined according to the edge pixel point.

13. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 6.

14. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-6.

15. A computer program product, comprising a computer program, which, when executed by a processor, implements the steps of the method according to any one of claims 1 to 6.

Images