Method, apparatus, controller and program product for displaying grid

Abstract

The invention relates to a method, an apparatus, a controller and a program product for displaying a grid. The method includes acquiring a panoramic image around a vehicle and determining a location of an obstacle based on the panoramic image. The method further includes displaying a plurality of grids based on the location of the obstacle, wherein the plurality of grids represent the location of the obstacle and a distance of the obstacle from the vehicle. According to the embodiment of the invention, the panoramic image can be converted into the grid related to the obstacle, so that the position of the obstacle and the distance between the obstacle and the vehicle can be visually displayed. Therefore, the embodiment of the invention does not need to carry out a large amount of measurement and calculation and logical operation on related distance parameters, thereby reducing the display cost of the obstacle, improving the display efficiency, and improving the use experience of a user.

Description

Technical Field

[0001] The embodiments of this disclosure generally relate to the field of intelligent driving, and more specifically to methods, apparatus, controllers, and program products for displaying grids. Background Technology

[0002] With the increasing number of vehicles and the increasingly complex parking environment, parking has gradually become a major problem for users. Traditional parking methods often rely on the user's driving experience and visual judgment, using simple reversing radar or cameras to detect obstacles behind the vehicle and remind the user to adjust their parking strategy through sound or visual means.

[0003] To enhance parking safety and convenience, Park Aid systems (PAS) are gaining increasing importance. PAS can generate real-time parking maps and plan parking routes, guiding users to control the vehicle or even take over parking altogether. As user parking needs continue to evolve, PAS is constantly being upgraded and improved to adapt to increasingly complex and changing parking environments. Summary of the Invention

[0004] Embodiments of this disclosure provide a method, apparatus, controller, and program product for displaying a grid.

[0005] In a first aspect of this disclosure, a method for displaying a grid is provided. The method includes acquiring a panoramic image of the area surrounding a vehicle. The method also includes determining the location of obstacles based on the panoramic image. Furthermore, the method includes displaying a plurality of grids based on the location of the obstacles, wherein the plurality of grids indicate the location of the obstacles and the distance between the obstacles and the vehicle.

[0006] In a second aspect of this disclosure, an apparatus for displaying a grid is provided. The apparatus includes a panoramic image acquisition module configured to acquire panoramic images of the area surrounding a vehicle. The apparatus also includes an obstacle location determination module configured to determine the location of obstacles based on the panoramic images. Furthermore, the apparatus includes a grid display module configured to display a plurality of grids based on the locations of the obstacles, wherein the plurality of grids indicate the location of the obstacles and the distance between the obstacles and the vehicle.

[0007] In a third aspect of this disclosure, a controller is provided. The controller includes at least one processor. The controller also includes memory coupled to the at least one processor and having instructions stored thereon, which, when executed by the at least one processor, cause the controller to perform the method provided according to the first aspect.

[0008] In a fourth aspect of this disclosure, a vehicle is provided. The vehicle includes a controller provided according to a third aspect of this disclosure.

[0009] In a fifth aspect of this disclosure, a machine program product is provided, comprising a machine program that is executed by a processor to implement the method provided in the first aspect.

[0010] In a sixth aspect of the disclosure, a machine-readable storage medium is provided. The machine-readable storage medium stores machine-executable instructions, which are executed by a processor to implement the method provided according to a first aspect of this disclosure.

[0011] It should be understood that the description in the Summary of the Invention section is not intended to limit the key or essential features of the embodiments of this disclosure, nor is it intended to restrict the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0012] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. In the drawings, the same or similar reference numerals denote the same or similar elements, wherein:

[0013] Figure 1A A schematic diagram of a scenario for displaying a risk zone between a vehicle and an obstacle, based on some relevant embodiments, is shown;

[0014] Figure 1B A schematic diagram of another scenario illustrating a risk zone between a vehicle and an obstacle, based on some relevant embodiments, is shown;

[0015] Figure 2 A schematic diagram of an example environment in which some embodiments of this disclosure may be implemented is shown;

[0016] Figure 3 Flowcharts of methods for displaying a grid, representing some embodiments of this disclosure, are shown;

[0017] Figure 4A A schematic diagram of a grid for displaying obstacles, representing some embodiments of this disclosure, is shown;

[0018] Figure 4B A schematic diagram of another grid for displaying obstacles is shown, representing some embodiments of this disclosure;

[0019] Figure 5A A schematic diagram of a grid for displaying collision risk levels, according to some embodiments of this disclosure, is shown;

[0020] Figure 5B A schematic diagram of the grid used in this disclosure to display obstacles and collision levels is shown;

[0021] Figure 6A A schematic diagram of a grid for displaying a local region is shown, representing some embodiments of this disclosure;

[0022] Figure 6B Schematic diagrams of some embodiments of this disclosure for displaying grids of obstacles and local areas are shown;

[0023] Figure 7A A schematic diagram of a grid used to display collision risk levels and local areas, according to some embodiments of this disclosure, is shown;

[0024] Figure 7B Schematic diagrams illustrating some embodiments of this disclosure for displaying obstacles, collision risk levels, and grids for local areas are shown;

[0025] Figure 8 Block diagrams of apparatus for displaying a grid, according to some embodiments of the present disclosure, are shown; and

[0026] Figure 9 A schematic block diagram of a controller according to some embodiments of the present disclosure is shown.

[0027] In all the accompanying figures, the same or similar reference numerals denote the same or similar elements. Detailed Implementation

[0028] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0029] In the description of embodiments of this disclosure, the term "comprising" and similar terms should be understood as open-ended inclusion, i.e., "including but not limited to". The term "based on" should be understood as "at least partially based on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The terms "first", "second", etc., may refer to different or the same objects. Other explicit and implicit definitions may also be included below.

[0030] In parking assist systems, users typically receive obstacle-related information alerts when obstacles are detected around the vehicle and approach it. Figure 1A , Figure 1B Schematic diagrams of scenarios 100A and 100B, respectively, of some related embodiments for displaying a risk area between a vehicle and an obstacle, are shown. (See reference) Figure 1A and Figure 1BThe parking assist system can display the vehicle and its surrounding area on the vehicle's infotainment interface. This area is divided into multiple sub-areas, such as sub-area D1 and sub-area D2. The parking assist system can then determine which sub-areas an obstacle falls into and display the obstacle and its corresponding sub-area on the infotainment interface. Furthermore, the parking assist system can also display risk areas between the vehicle and obstacles on the infotainment interface, including risk areas S1 to S10. In this way, the user can observe the position of obstacles and the distance between obstacles and the vehicle through the infotainment interface.

[0031] To display the above content on the vehicle's infotainment interface, the relevant methods require geometrically calculating multiple distance parameters between points and lines on the obstacle's surface and the vehicle. These distance parameters are then used to determine the sub-region where the obstacle is located. For example, refer to... Figure 1A Taking a key point P of an obstacle as an example, the parking assistance system needs to calculate the distance between key point P and key points A and B of the vehicle. Then, through distance calculations, it can determine that the obstacle is located in sub-region D6, and also determine the risk area S3 formed by the vertical distance between the obstacle's key point P and the vehicle. However, the calculation of distance parameters in this method is complex, and it involves a large amount of derivation calculations and logical judgments in determining the sub-region to which the obstacle belongs and calculating the distance between the obstacle and the vehicle. Therefore, displaying the position of the obstacle and the distance between the obstacle and the vehicle in this method is costly.

[0032] Therefore, embodiments of this disclosure provide a method for displaying a grid. This method acquires and detects panoramic images of the area surrounding a vehicle to determine the positions of obstacles within the panoramic image. Then, based on the obstacle positions, the method displays the panoramic image in the form of a grid, allowing the user to directly observe the obstacle positions and their distances from the vehicle based on the displayed grid. In embodiments of this disclosure, by converting the panoramic image into an obstacle-related grid, the positions of obstacles and their distances from the vehicle can be displayed intuitively. Therefore, embodiments of this disclosure eliminate the need for extensive calculations and logical operations on related distance parameters, thereby reducing display costs, improving display efficiency, and enhancing the user experience.

[0033] Figure 2A schematic diagram of an example environment 200 that can be implemented in some embodiments of the present disclosure is shown. In some embodiments, the example environment 200 describes a parking scenario of a vehicle 202, and includes multiple obstacles, such as obstacle 2041, obstacle 2042, and obstacle 2043. In some embodiments of the present disclosure, obstacles refer to targets that may collide with the vehicle 202 during parking, including but not limited to walls, pillars, pedestrians, fences, and animals. The environment 200 in this embodiment includes, but is not limited to, roads, parking lots, and other similar scenarios.

[0034] In some embodiments, the vehicle 202 includes an electronic device 206, which includes a camera 208, a controller 210, and a screen 218. In some embodiments, the camera 208, depending on its arrangement, includes, but is not limited to, a front-view camera, a rear-view camera, a surround-view camera, and a side-view camera. The controller 210 includes, but is not limited to, an autonomous driving domain controller, a body controller, a vehicle controller, a parking assistance domain controller, or any combination of two or more of the above controllers. The screen 218 includes, but is not limited to, a central control display screen, a multimedia entertainment screen, an integrated screen, a rearview mirror screen, and a projection screen, such as a head-up display (HUD) projection screen.

[0035] In some embodiments, camera 208 is used to acquire a panoramic image 212 around vehicle 202. In some embodiments, camera 208 includes multiple cameras arranged around the vehicle body, which can acquire local images from multiple directions. Then, camera 208 can synthesize the panoramic image 212 based on the multiple local images and parameters associated with the multiple cameras. Alternatively or additionally, camera 208 can also send the multiple local images to controller 210, which will then synthesize the panoramic image 212. In some embodiments, the panoramic image 212 is a bird's-eye view of the area around the vehicle.

[0036] In some embodiments, the controller 210 acquires a panoramic image 212 and performs image detection on the panoramic image 212 to determine the location 214 of an obstacle. In some embodiments, the controller 210 can determine the location 214 of the obstacle through image segmentation, object detection, or other methods. Then, the controller 210 can display the panoramic image 212 on the screen 218 in the form of a grid 216 according to the location 214 of the obstacle, wherein the grid 216 can be displayed differently depending on whether it is related to the obstacle. For example, the grid inside the obstacle is highlighted, and the grid outside the obstacle is left blank. In this way, the user can observe the location of the obstacle in the environment 200 according to the grid 216 displayed on the screen 218. Furthermore, since the size of the grid 216 is often fixed, for example, 0.5*0.5m. 2 Therefore, users can also directly observe the distance between obstacles and vehicles 202 based on the grid 216 displayed on screen 218.

[0037] In the embodiments of this disclosure, the controller 210 can intuitively display the position of the obstacle in the environment 200 and the distance between the obstacle and the vehicle 202 by converting the panoramic image 212 into a grid 216 related to the position 214 of the obstacle. Therefore, the embodiments of this disclosure do not require a large amount of calculation and logical operation on the distance parameter between the obstacle and the vehicle 202, thereby reducing display costs, improving display efficiency, and enhancing the user experience.

[0038] It should be understood that the architecture and functionality in example environment 100 are described for illustrative purposes only and do not imply any limitation on the scope of this disclosure. Embodiments of this disclosure can also be applied to other environments with different structures and / or functionalities.

[0039] The following will combine Figures 3 to 9 The process according to embodiments of this disclosure is described in detail. For ease of understanding, the specific data mentioned in the following description are exemplary and not intended to limit the scope of this disclosure. It should be understood that the embodiments described below may also include additional actions not shown and / or actions shown may be omitted, and the scope of this disclosure is not limited in this respect.

[0040] Figure 3 A flowchart illustrating a method 300 for displaying a grid, according to some embodiments of the present disclosure, is shown. In some embodiments, in... Figure 2In the example environment 200 shown, method 300 can be executed by controller 210. In some embodiments, controller 210 includes, but is not limited to, an autonomous driving domain controller, a body controller, a vehicle controller, a parking assistance domain controller, or a combination of any two or more of the above controllers. It should be understood that although the following description uses controller 210 as the executing entity, method 300 can also be executed by other devices. Method 300 may also include additional actions not shown and / or actions shown may be omitted; the scope of this disclosure is not limited in this respect.

[0041] At point 302, a panoramic image of the vehicle's surroundings is acquired. In some embodiments, the controller 210 acquires a panoramic image 212 transmitted by the camera 208. Alternatively or additionally, the controller 210 acquires partial images captured by multiple cameras in the camera 208 and synthesizes the panoramic image 212 based on the multiple partial images. It should be understood that the panoramic image 212 includes not only the vehicle 202, but also the areas around the vehicle 202 that it may travel to, and any obstacles that may exist in those areas.

[0042] At position 304, the location of the obstacle is determined based on the panoramic image. In some embodiments, after acquiring the panoramic image 212, the controller 210 performs image detection on the panoramic image 212 to determine the location 214 of the obstacle. In some embodiments, the controller 210 can determine the location 214 of the obstacle through image segmentation, object detection, or other methods. The location 214 of the obstacle is used to determine the grid related to the obstacle within the grid of the panoramic image.

[0043] At position 306, multiple grids are displayed based on the obstacle's location, indicating the obstacle's position and distance from the vehicle. In some embodiments, the controller 210 can display a panoramic image 212 on screen 218 as a grid 216 based on the obstacle's location 214, where the grid 216 can be differentiated according to whether it is related to the obstacle. For example, grids inside the obstacle are highlighted, while grids outside the obstacle are left blank. Thus, the user can observe the obstacle's position in the environment 200 based on the grid 216 displayed on screen 218. Furthermore, since the size of the grid 216 is often fixed, the user can also directly observe the distance between the obstacle and the vehicle 202 based on the grid 216 displayed on screen 218.

[0044] This disclosure provides a method for displaying a grid. The method acquires and detects panoramic images of the area around a vehicle to determine the positions of obstacles within the panoramic image. Then, based on the obstacle positions, the method displays the panoramic image as a grid, allowing the user to directly observe the obstacle positions and their distances from the vehicle. In this disclosure, by converting the panoramic image into an obstacle-related grid, the positions of obstacles and their distances from the vehicle can be displayed intuitively. Therefore, this disclosure eliminates the need for extensive calculations and logical operations on related distance parameters, thereby reducing display costs, improving display efficiency, and enhancing the user experience.

[0045] In some embodiments, the controller 210 can monitor whether a user intends to park. When a user intends to park, such as when the user performs a parking operation, or when it is determined from the analysis results of the driving scenario and user operation that the user is preparing to park, the controller 210 can acquire a panoramic image 212 of the area around the vehicle 202 via the camera 206. In some embodiments, the size of each grid can be set based on the panoramic image 212. For example, a grid can be defined as 16*16 pixels. In this way, the grid size can be quickly determined based on the panoramic image 212. Alternatively or additionally, the grid size can be set based on a fixed size, such as setting the grid size to 0.5*0.5m. 2 .

[0046] Figure 4A A schematic diagram of a grid 400A for displaying obstacles, according to some embodiments of this disclosure, is shown. In some embodiments, the grid 400A includes a highlighted grid 402 and a blank grid 404. The highlighted grid 402 is used to display the range associated with the obstacle, and the blank grid 404 is used to display the range outside the obstacle. In some embodiments, the controller 210 can determine whether a portion of the grid falls within the range of the obstacle based on the location of the obstacle. If a portion of the grid in the grid 400A falls within the range of the obstacle, the controller 210 determines that the grid is associated with the obstacle and displays the grid in a highlighted manner. If no grid in the grid 400A falls within the range of the obstacle, the controller 210 determines that the grid is unrelated to the obstacle and displays the grid in a blank manner.

[0047] refer to Figure 4AUsers can determine the location of obstacles based on the highlighted grid 402. Furthermore, since each grid has a fixed size, users can estimate the distance between the obstacle and the vehicle based on the number of grids between the highlighted grid 402 and the vehicle area 406. In this way, the controller 210 can quickly determine whether a grid is related to an obstacle based on its location, and then display the grid based on the result of whether the grid is related to an obstacle, thereby improving the grid display efficiency.

[0048] Figure 4B A schematic diagram of another grid 400B for displaying obstacles is shown, representing some embodiments of this disclosure. In some embodiments, the controller 210 may pre-establish a vehicle coordinate system o for displaying the grid occupied by obstacles. e xy (which can be called the first coordinate system) and grid coordinate system o c mn (which can be called the second coordinate system). Among them, the vehicle coordinate system o... e The units for the x-axis and y-axis in xy are meters (m), and the grid coordinate system is o. c The unit in mn is 1 grid. (Automotive coordinate system o) e The origin of xy e In grid coordinate system o c The coordinate in mn is o c (ec_i, ec_j). It should be understood that this is based on the vehicle's coordinate system o. e The origin of xy e In grid coordinate system o c The coordinates o in mn c (ec_i, ec_j) and the side length of the grid (e.g., set to 0.5m) can be used to establish the vehicle coordinate system o. e xy and grid coordinate system o c The conversion relationship between m and n.

[0049] Then, the controller 210, based on the position of the obstacle in the panoramic image 212, in the vehicle coordinate system o e Determine the coordinates of the obstacle in xy. e (o_x, o_y) (which can be called the first coordinates). Further, the controller 210 uses the vehicle coordinate system o... e xy and grid coordinate system o c The transformation relationship between mn will transform the vehicle coordinate system o. e The coordinates of the obstacle in xy are o e (o_x,o_y) are transformed to the grid coordinate system o c The coordinates of mn are o c(ec_i–o_x / 0.5, ec_j–o_y / 0.5) (which can be called the second coordinates), and highlight the grid corresponding to this transformed coordinate. For example, when the transformed coordinate is (2,2), the grid corresponding to the second row and second column is highlighted. In this way, the controller 210 can be based on the vehicle coordinate system o e After determining the obstacle's position coordinates (x, y), quickly transform the obstacle's position coordinates to the grid coordinate system (o). c Under mn, it can then be determined according to the grid coordinate system o c The coordinates under mn are used to locate the corresponding grid and highlight it, thereby improving the display efficiency of obstacles in the grid.

[0050] In some embodiments, the controller 210 can determine the collision risk level of an obstacle within a grid colliding with the vehicle 202 based on the distance between the grid and the vehicle area 406 occupied by the vehicle 202. It should be understood that the closer the grid is to the vehicle area 406, the higher the collision risk. The controller 210 can then display the grid 402 associated with the obstacle and the associated collision risk level. In this way, the user can intuitively determine the collision risk of each grid based on the displayed content of the obstacle-related grid 402, thereby improving safety during parking.

[0051] Figure 5A A schematic diagram of a grid 500A for displaying collision risk levels according to some embodiments of this disclosure is shown. In grid 500A, if the distance between the grid and the vehicle area 406 is 0 (referred to as the first distance), meaning that a portion of the grid is located within the vehicle area 406, then the controller 210 determines the collision risk level of the grid (referred to as the first grid) to be level 1 (referred to as the first level), for example, the collision risk level of grid (4,4) is level 1. If the distance between the grid and the vehicle area 406 is 1 (referred to as the second distance), meaning that a portion of the grid adjacent to the grid is located within the vehicle area 406, then the controller 210 determines the collision risk level of the grid (referred to as the second grid) to be level 2 (referred to as the second level), for example, the collision risk level of grid (4,3) is level 2. Similarly, a grid with a collision risk level of level 3, such as grid (3,3), and a grid with a collision risk level of 0, such as grid (0,0), can also be determined based on distance, which will not be elaborated further in this embodiment. It should be understood that the method of determining the collision risk level in this embodiment is merely one example.

[0052] In some embodiments, after determining the corresponding collision risk level based on the distance between the grid and the vehicle area 406, the numerical value corresponding to the collision risk level (referred to as the first numerical value) can be further determined. For example, the numerical value corresponding to level 1 is 1, and the numerical value corresponding to level 2 is 2. Then, based on the numerical value of each grid and the position of each grid in the entire panoramic area (referred to as the rectangular area), a risk level matrix E1(i,j) (referred to as the first matrix) is established, where i and j represent the m-axis coordinate and n-axis coordinate of the grid, respectively.

[0053] Then, refer to Figure 4B The controller 210 can determine a value (referred to as a second value) indicating whether a grid is associated with an obstacle based on whether a portion of the grid is within the obstacle's area. If a portion of the grid is within the obstacle's area, the value of that grid is set to 1, for example, grid (8,2) has a value of 1. If the grid is not within the obstacle's area at all, the value of that grid is set to 0, for example, grid (10,1) has a value of 0. Then, the controller 210 establishes an obstacle occupancy matrix O(i,j) (referred to as a second matrix) based on the value of each grid and the position of each grid in the entire panoramic area.

[0054] Figure 5B A schematic diagram of the grid 500B used to display obstacles and collision levels according to this disclosure is shown. After establishing the risk level matrix E1(i,j) and the obstacle occupancy matrix O(i,j), the controller 210 can calculate the obstacle feature matrix F1(i,j) = E1(i,j) * O(i,j), where "*" indicates multiplication of elements at the same position in the matrix. After obtaining the obstacle feature matrix F1(i,j), the grid 500B can be displayed based on this obstacle feature matrix F1(i,j), and the grids in the grid 500B are displayed based on the result of the multiplication. In this way, the controller 210 can simultaneously display whether a grid is related to an obstacle and the collision risk level corresponding to that grid. For example, the grid (8,2) can represent both the location of the obstacle and the corresponding collision risk level, thereby improving the accuracy and comprehensiveness of the grid display information.

[0055] In some embodiments, the controller 210 can divide multiple grids in the panoramic area into multiple local regions according to the orientation of the vehicle 202, for example, dividing the panoramic area into four local regions according to the orientation of front, back, left, and right. Then, the controller 210 determines the local region to which each grid belongs and displays the grid 402 associated with the obstacle and the local region of each grid in grid 402. In this way, the user can intuitively observe the local region and orientation of the grid 402 based on the displayed content of the obstacle-related grid 402, thereby improving safety during the parking process.

[0056] Figure 6A A schematic diagram of a grid 600A for displaying local regions, according to some embodiments of the present disclosure, is shown. In grid 600A, the panoramic area is divided into eight local regions according to eight directions of the vehicle 202: front left, upper left, lower left, rear left, front right, upper right, lower right, and rear right, sequentially including local regions M1 to M8. Then, the controller 210 determines a numerical value (which may be called a third value) representing the local region to which each grid belongs. For example, if grid (2,3) belongs to local region M1, then the value of grid (2,3) is 1; if grid (8,2) belongs to local region M8, then the value of grid (8,2) is 8. Further, the controller 210 establishes a local region attribution matrix E2(i,j) (which may be called a third matrix) based on the value of each grid and the position of each grid in the entire panoramic area.

[0057] Then, refer to Figure 4B The controller 210 can determine a value (referred to as a second value) indicating whether a grid is associated with an obstacle based on whether a portion of the grid is within the obstacle's area. If a portion of the grid is within the obstacle's area, the value of that grid is set to 1, for example, grid (8,2) has a value of 1. If the grid is not within the obstacle's area at all, the value of that grid is set to 0, for example, grid (10,1) has a value of 0. Then, the controller 210 establishes an obstacle occupancy matrix O(i,j) (referred to as a second matrix) based on the value of each grid and the position of each grid in the entire panoramic area.

[0058] Figure 6B A schematic diagram of a grid 600B for displaying obstacles and local regions, according to some embodiments of this disclosure, is shown. After establishing the local region attribution matrix E2(i,j) and the obstacle occupancy matrix O(i,j), the controller 210 can calculate the obstacle feature matrix F2(i,j) = E2(i,j) * O(i,j), where "*" indicates multiplication of elements at the same position in the matrix. After obtaining the obstacle feature matrix F2(i,j), the grid 600B can be displayed based on this obstacle feature matrix F2(i,j), and the grids in the grid 600B are displayed based on the result of the multiplication. In this way, the controller 210 can simultaneously display whether a grid is related to an obstacle and the local region to which the grid belongs. For example, the grid (8,2) can represent both an obstacle and the local region M2 to which it belongs, thereby improving the accuracy and comprehensiveness of the grid display information.

[0059] Figure 7AA schematic diagram of a grid 700A for displaying collision risk levels and local regions, according to some embodiments of this disclosure, is shown. In some embodiments, the controller 210 can merge the aforementioned grids 500A and 500B to determine the numerical value (referred to as the fourth numerical value) corresponding to each grid in grid 700A. Specifically, the controller 210 uses the numerical value representing the collision risk level of the grid as the units digit and the numerical value representing the local region to which the grid belongs as the tens digit, thereby obtaining the numerical value corresponding to each grid. For example, if the collision risk level of grid (3,3) is 3 and the local region to which it belongs is 1, then the numerical value of grid (3,3) is 13. If the collision risk level of a grid is 0, the controller 210 can directly set the numerical value corresponding to that grid to 0. Then, the controller 210 establishes a matrix E(i,j) (referred to as the fourth matrix) representing the risk level and local region affiliation based on the numerical value of each grid and the position of each grid in the entire panoramic area.

[0060] Then, refer to Figure 4B The controller 210 can determine a value (referred to as a second value) indicating whether a grid is associated with an obstacle based on whether a portion of the grid is within the obstacle's area. If a portion of the grid is within the obstacle's area, the value of that grid is set to 1, for example, grid (8,2) has a value of 1. If the grid is not within the obstacle's area at all, the value of that grid is set to 0, for example, grid (10,1) has a value of 0. Then, the controller 210 establishes an obstacle occupancy matrix O(i,j) (referred to as a second matrix) based on the value of each grid and the position of each grid in the entire panoramic area.

[0061] Figure 7B A schematic diagram of a grid 700B for displaying obstacles, collision risk levels, and local regions, according to some embodiments of this disclosure, is shown. After establishing a matrix E(i,j) representing the risk level and local region affiliation, and an obstacle occupancy matrix O(i,j), the controller 210 can calculate the obstacle feature matrix F(i,j) = E(i,j) * O(i,j), where "*" indicates multiplication of elements at the same position in the matrix. After obtaining the obstacle feature matrix F(i,j), the grid 700B can be displayed based on this matrix, with the grids in the grid 700B displayed based on the result of the multiplication. In this way, the controller 210 can simultaneously display whether a grid is associated with an obstacle, the collision risk level corresponding to the grid, and the local region to which the grid belongs. For example, the grid (8,2) can represent the location of an obstacle, the corresponding collision risk level of level 3, and the local region M3 to which it belongs, thereby improving the accuracy and comprehensiveness of the grid display information.

[0062] In some embodiments, after obtaining the obstacle feature matrix F(i,j), if it is necessary to determine the collision risk level of a local region in grid 700B, the controller 210 can extract the units digit of the maximum value of the grid in the local region and use the collision risk level corresponding to the units digit as the collision risk level of the local region. In some embodiments, the units digit of the maximum value can be extracted by a modulo operation. For example, in local region M3, if the maximum value of the grid is 33, the controller can divide 33 by 10 and perform a modulo operation to obtain the units digit 3, thereby determining the collision risk level of local region M3 as level 3.

[0063] Figure 8 A block diagram of an apparatus 800 for displaying a grid, according to some embodiments of the present disclosure, is shown. The apparatus 800 includes a panoramic image acquisition module 802 configured to acquire panoramic images of the area surrounding a vehicle. The apparatus 800 also includes an obstacle location determination module 804 configured to determine the location of obstacles based on the panoramic images. Furthermore, the apparatus 800 includes a grid display module 806 configured to display a plurality of grids based on the locations of the obstacles, wherein the plurality of grids indicate the location of the obstacles and the distance between the obstacles and the vehicle.

[0064] In some embodiments, the grid display module 806 is further configured to: determine, based on the location of the obstacle, whether at least a portion of a grid in a plurality of grids is within the range of the obstacle; and display the grid, indicating whether at least a portion is within the range of the obstacle and the distance between the grid and the vehicle.

[0065] In some embodiments, the grid display module 806 is further configured to: determine the levels of multiple grids based on the distances of multiple grids to the vehicle, wherein the levels indicate the collision risk of the vehicle; and display the grids, indicating whether at least a portion of the grid is within the range of an obstacle and the grid level.

[0066] In some embodiments, the rectangular area around the vehicle is divided into multiple grids according to a preset granularity, and the grid display module 806 is further configured to: determine multiple first values ​​of the indication level based on the distance between the multiple grids and the vehicle; and establish a first matrix based on the multiple first values ​​corresponding to the multiple grids and the positions of the multiple grids in the rectangular area.

[0067] In some embodiments, the grid display module 806 is further configured to: determine a plurality of second values ​​indicating whether at least a portion of a plurality of grids is within the range of an obstacle; establish a second matrix based on the plurality of second values ​​corresponding to the plurality of grids and the positions of the plurality of grids in a rectangular region; and display the grids based on the first matrix and the second matrix, wherein the grids indicate the product of the first value and the second value.

[0068] In some embodiments, the plurality of grids includes a first grid and a second grid, and the grid display module 806 is further configured to: determine a first level of the first grid when the distance between the first grid and the vehicle is a first distance, the first distance indicating that at least a portion of the first grid is within the range of the vehicle; and determine a second level of the second grid when the distance between the second grid and the vehicle is a second distance, the second distance indicating that at least a portion of the grid adjacent to the second grid is within the range of the vehicle.

[0069] In some embodiments, the grid display module 806 is further configured to: determine the local area to which a plurality of grids belong, the plurality of grids being divided into a plurality of local areas according to the vehicle's orientation; and display the grids, the grids indicating whether at least a portion is within the range of an obstacle, the distance between the grids and the vehicle, and the local area to which the grids belong.

[0070] In some embodiments, the rectangular area around the vehicle is divided into multiple grids according to a preset granularity, and the grid display module 806 is further configured to: determine multiple third values ​​indicating the local area to which the multiple grids belong; and establish a third matrix based on the multiple third values ​​corresponding to the multiple grids and the positions of the multiple grids in the rectangular area.

[0071] In some embodiments, the grid display module 806 is further configured to: determine a plurality of second values ​​indicating whether at least a portion of a plurality of grids is within the range of an obstacle; establish a second matrix based on the plurality of second values ​​corresponding to the plurality of grids and the positions of the plurality of grids in a rectangular region; and display the grids based on the second matrix and the third matrix, wherein the grids indicate the product of the second values ​​and the third values, and indicate the distance between the grids and the vehicle.

[0072] In some embodiments, the rectangular area surrounding the vehicle is divided into multiple grids according to a preset granularity, and the grid display module 806 is further configured to: determine multiple fourth values ​​corresponding to the multiple grids, wherein the multiple fourth values ​​indicate the collision risk level of the multiple grids and the local area to which they belong, and the level is related to the distance between the multiple grids and the vehicle; establish a fourth matrix based on the multiple fourth values ​​corresponding to the multiple grids and the positions of the multiple grids in the rectangular area; determine multiple second values ​​indicating whether at least a portion of the multiple grids are within the range of an obstacle; establish a second matrix based on the multiple second values ​​corresponding to the multiple grids and the positions of the multiple grids in the rectangular area; and display the grids based on the second matrix and the fourth matrix, wherein the grids indicate the product of the second values ​​and the fourth values.

[0073] In some embodiments, the grid display module 806 is further configured to: determine a first coordinate of an obstacle in a first coordinate system, the first coordinate system being related to a vehicle, based on the location of the obstacle; transform the first coordinate to a second coordinate based on a transformation relationship between the first coordinate system and a second coordinate system, the second coordinate system being related to a plurality of grids; and display the grids related to the second coordinates.

[0074] In some embodiments, the panoramic image acquisition module 802 is further configured to: determine whether a user's parking intention is detected; and if a user's parking intention is detected, acquire a panoramic image of the area around the vehicle.

[0075] It is understood that the apparatus 800 of this disclosure can achieve at least one of the many advantages that the methods or processes described above can achieve. For example, the apparatus 800 can acquire and detect panoramic images of the area around a vehicle to determine the position of obstacles in the panoramic image. Then, the apparatus can display the panoramic image in a grid format based on the position of the obstacles, allowing the user to directly observe the position of the obstacles and their distance from the vehicle based on the displayed grid. In embodiments of this disclosure, by converting the panoramic image into an obstacle-related grid, the position of the obstacles and their distance from the vehicle can be displayed intuitively. Therefore, embodiments of this disclosure do not require extensive calculations and logical operations on the relevant distance parameters, thereby reducing display costs and improving display efficiency.

[0076] Figure 9 A schematic block diagram of a controller according to some embodiments of the present disclosure is shown. Figure 9 A schematic block diagram of a controller 900 that can be used to implement embodiments of the present disclosure is shown. In some embodiments, the controller 900 is used to implement... Figure 2 The controller 210 is shown in the example environment 200. Figure 9 As shown, the controller 900 includes a processor 901, which can perform various appropriate actions and processes based on machine program instructions loaded into random access memory (RAM) 903 according to machine program instructions stored in read-only memory (ROM) 902. The RAM 903 may also store various programs and data required for the operation of the controller 900. The processor 901, ROM 902, and RAM 903 are interconnected via bus 904. An input / output (I / O) interface 905 is also connected to bus 904.

[0077] The various processes and procedures described above, such as method 300, can be executed by processor 901. For example, in some embodiments, method 300 may be implemented as a machine software program tangibly contained in a machine-readable medium. In some embodiments, part or all of the machine program may be loaded into and / or mounted onto controller 900 via ROM 902. When the machine program is loaded into RAM 903 and executed by processor 901, one or more actions of method 300 described above may be performed.

[0078] This disclosure can be a method, apparatus, system, and / or machine program product. A machine program product may include a machine-readable storage medium loaded with machine-readable program instructions for performing various aspects of this disclosure.

[0079] Machine-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Machine-readable storage media can be, for example—but not limited to—electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of machine-readable storage media include: random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), and any suitable combination of the foregoing. As used herein, machine-readable storage media is not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0080] The machine-readable program instructions described herein can be downloaded from machine-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to external machines or external storage devices. The network may include copper cables, fiber optic cables, wireless transmission, routers, firewalls, switches, gateway machines, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the machine-readable program instructions from the network and forwards them to the machine-readable storage media within the respective computing / processing device.

[0081] Machine program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The machine-readable program instructions may be executed entirely on the user machine, partially on the user machine, as a stand-alone software package, partially on the user machine and partially on a remote machine, or entirely on a remote machine or server. In cases involving remote machines, the remote machine may be connected to the user machine via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external machine (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing state information from the machine-readable program instructions to implement various aspects of this disclosure.

[0082] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and machine program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by machine-readable program instructions.

[0083] These machine-readable program instructions can be provided to the processing unit of a general-purpose machine, a special-purpose machine, or other programmable data processing apparatus to produce a machine such that, when executed by the processing unit of the machine or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These machine-readable program instructions can also be stored in a machine-readable storage medium that causes a machine, programmable data processing apparatus, and / or other device to operate in a particular manner. Thus, the machine-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0084] Machine-readable program instructions may also be loaded onto a machine, other programmable data processing apparatus, or other equipment to cause a series of operational steps to be performed on the machine, other programmable data processing apparatus, or other equipment to produce a machine-implemented process, thereby causing the instructions executed on the machine, other programmable data processing apparatus, or other equipment to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0085] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and machine program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and machine instructions.

[0086] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or technical improvements to the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A method (300) for displaying a grid, comprising: acquiring (302) a panoramic image of a surrounding of a vehicle; determining (304) a position of an obstacle based on the panoramic image; and displaying (306) a plurality of grids based on the position of the obstacle, wherein the plurality of grids indicate the position of the obstacle and a distance of the obstacle from the vehicle. 2.The method (300) of claim 1, wherein displaying (306) the plurality of grids based on the position of the obstacle comprises: determining whether at least a portion of a grid in the plurality of grids is within a range of the obstacle based on the position of the obstacle; and displaying the grid indicating whether the at least a portion is within the range of the obstacle and a distance of the grid from the vehicle. 3.The method (300) of claim 2, wherein displaying the grid comprises: determining a level of the plurality of grids based on the distance of the plurality of grids from the vehicle, wherein the level indicates a collision risk of the vehicle; and displaying the grid indicating whether the at least a portion is within the range of the obstacle and the level of the grid. 4.The method (300) of claim 3, wherein a rectangular area of the surrounding of the vehicle is divided into the plurality of grids according to a preset granularity, and determining the level of the plurality of grids based on the distance of the plurality of grids from the vehicle comprises: determining a plurality of first values indicating the level based on the distance of the plurality of grids from the vehicle; and establishing a first matrix based on the plurality of first values corresponding to the plurality of grids and positions of the plurality of grids in the rectangular area. 5.The method (300) of claim 4, wherein displaying the grid comprises: determining a plurality of second values indicating whether at least a portion of the plurality of grids is within the range of the obstacle; establishing a second matrix based on the plurality of second values corresponding to the plurality of grids and the positions of the plurality of grids in the rectangular area; and displaying the grid indicating a product result of the first values and the second values based on the first matrix and the second matrix. 6.The method (300) of claim 3, wherein the plurality of grids comprises a first grid and a second grid, and determining the level of the plurality of grids based on the distance of the plurality of grids from the vehicle comprises: determining a first level of the first grid in a case that a distance of the first grid from the vehicle is a first distance, the first distance indicating that at least a portion of the first grid is within a range of the vehicle; and determining a second level of the second grid in a case that a distance of the second grid from the vehicle is a second distance, the second distance indicating that at least a portion of a grid adjacent to the second grid is within the range of the vehicle. 7.The method (300) of claim 2, wherein displaying the grid comprises: determining a local area to which the plurality of grids belong, the plurality of grids being divided into a plurality of local areas according to a bearing of the vehicle; and displaying the grid indicating whether the at least a portion is within the range of the obstacle and a distance of the grid from the vehicle based on the local area to which the plurality of grids belong. ​ ​ ​ ​ ​ ​ displaying the grids, the grids indicating whether at least a portion of the grids is within a range of the obstacle, distances of the grids from the vehicle, and the local areas to which the grids belong.

8. The method (300) of claim 7, wherein the rectangular area around the vehicle is divided into the plurality of grids according to a preset granularity, and determining the local areas to which the plurality of grids belong comprises: determining a plurality of third values indicating the local areas to which the plurality of grids belong; and establishing a third matrix based on the plurality of third values corresponding to the plurality of grids and positions of the plurality of grids in the rectangular area.

9. The method (300) of claim 8, wherein displaying the grids comprises: determining a plurality of second values indicating whether at least a portion of the plurality of grids is within a range of the obstacle; establishing a second matrix based on the plurality of second values corresponding to the plurality of grids and positions of the plurality of grids in the rectangular area; and displaying the grids based on the second matrix and the third matrix, the grids indicating multiplication results of the second values and the third values, and distances of the grids from the vehicle.

10. The method (300) of claim 7, wherein the rectangular area around the vehicle is divided into the plurality of grids according to a preset granularity, and displaying the grids comprises: determining a plurality of fourth values corresponding to the plurality of grids, wherein the plurality of fourth values indicate levels of collision risks of the plurality of grids and the local areas to which the plurality of grids belong, the levels being related to distances of the plurality of grids from the vehicle; establishing a fourth matrix based on the plurality of fourth values corresponding to the plurality of grids and positions of the plurality of grids in the rectangular area; determining a plurality of second values indicating whether at least a portion of the plurality of grids is within a range of the obstacle; establishing a second matrix based on the plurality of second values corresponding to the plurality of grids and positions of the plurality of grids in the rectangular area; and displaying the grids based on the second matrix and the fourth matrix, the grids indicating multiplication results of the second values and the fourth values.

11. The method (300) of claim 1, wherein displaying the plurality of grids based on the position of the obstacle comprises: determining a first coordinate of the obstacle in a first coordinate system based on the position of the obstacle, the first coordinate system being related to the vehicle; converting the first coordinate to a second coordinate based on a conversion relationship between the first coordinate system and a second coordinate system, the second coordinate system being related to the plurality of grids; and displaying the grids related to the second coordinate.

12. The method (300) of any one of claims 1 to 11, wherein acquiring (302) the panoramic image around the vehicle comprises: determining whether a parking intention of a user is detected; and acquiring the panoramic image around the vehicle in a case where the parking intention of the user is detected.

13. An apparatus (800) for displaying grids, comprising: a panoramic image acquisition module (802) configured to acquire a panoramic image around a vehicle; ​ ​ an obstacle position determination module (804) configured to determine a position of an obstacle based on the panoramic image; and a grid display module (806) configured to display a plurality of grids based on the position of the obstacle, wherein the plurality of grids indicate the position of the obstacle and a distance of the obstacle from the vehicle.

14. A controller (210) comprising: at least one processor; and a memory coupled to the at least one processor and having stored thereon instructions which, when executed by the at least one processor, cause the controller to perform the method according to any one of claims 1 to 12.

15. A machine program product comprising a machine program which is executed by a processor to implement the method according to any one of claims 1 to 12.

Images