Image-based map generation method and device, equipment and storage medium

Abstract

The invention relates to an image-based map generation method, apparatus and device, and a storage medium. The method comprises the steps of obtaining an environment image of an environment where a vehicle is located and navigation information of the vehicle; wherein the navigation information comprises the driving direction of the vehicle from the starting point to the terminal point; determining an initial map of the environment where the vehicle is located according to the environment image; wherein the initial map represents static information in an environment where the vehicle is located; determining a target map of the environment where the vehicle is located according to the initial map and the navigation information; wherein the target map represents static information in an environment where the vehicle is located and information of a vehicle driving path. The accurate map is provided for the user according to the visual image, so that the vehicle is helped to perform intelligent driving.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving technology, and in particular to an image-based map generation method, device, equipment and storage medium. Background Art

[0002] With the development of autonomous driving technology, vehicles need to be automatically controlled based on maps of the environment when driving, and the accuracy of the map will affect the effect of autonomous driving.

[0003] In complex urban roads, high-precision maps may be missing due to issues such as geographic information management. Without high-precision maps, it is difficult for vehicles to perform accurate and safe autonomous driving. Therefore, it is necessary to generate accurate maps in the absence of high-precision maps to assist vehicles in planning and control. Summary of the invention

[0004] The purpose of the present invention is to provide an image-based map generation method, device, equipment and storage medium to achieve map generation without high-precision maps and improve the effect of autonomous driving.

[0005] In a first aspect, the present invention provides an image-based map generation method, characterized by comprising:

[0006] Acquire an environmental image of the environment in which the vehicle is located and navigation information of the vehicle; wherein the navigation information includes the driving direction of the vehicle from a starting point to an end point;

[0007] Determine an initial map of the environment in which the vehicle is located according to the environment image; wherein the initial map represents static information in the environment in which the vehicle is located;

[0008] A target map of the environment where the vehicle is located is determined according to the initial map and the navigation information; wherein the target map represents static information in the environment where the vehicle is located and information on the vehicle's driving path.

[0009] In a second aspect, the present invention provides an image-based map generation device, characterized in that it includes:

[0010] An information acquisition module, used to acquire an environmental image of the environment in which the vehicle is located and navigation information of the vehicle; wherein the navigation information includes the driving direction of the vehicle from the starting point to the end point;

[0011] An initial determination module, used to determine an initial map of the environment where the vehicle is located according to the environment image; wherein the initial map represents static information in the environment where the vehicle is located;

[0012] The target determination module is used to determine a target map of the environment where the vehicle is located based on the initial map and the navigation information; wherein the target map represents static information in the environment where the vehicle is located and information on the vehicle's driving path.

[0013] In a third aspect, the present invention provides an electronic device, comprising: a processor, and a memory communicatively connected to the processor;

[0014] The memory stores computer-executable instructions;

[0015] The processor executes the computer-executable instructions stored in the memory to implement the method according to the first aspect.

[0016] In a fourth aspect, the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, they are used to implement the method described in the first aspect.

[0017] In a fifth aspect, the present invention provides a computer program product, comprising a computer program, which implements the method described in the first aspect when executed by a processor.

[0018] The present invention provides an image-based map generation method, device, equipment and storage medium, which can determine the approximate driving direction of the vehicle by acquiring the navigation information of the vehicle. According to the environmental image, the static information in the environment where the vehicle is located is determined, so as to obtain the initial image of the environment where the vehicle is located, that is, the initial image only contains static information. According to the initial map and navigation information, the relevant information of the vehicle's driving path is determined, that is, the topological information is obtained, and the static information and the topological information are combined to obtain the target map. In the absence of a high-precision map, an accurate map containing static information and topological information is provided to the user, which assists the vehicle in automatic driving, improves the accuracy and safety of automatic driving, and enhances the user's driving experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0020] Figure 1 A schematic diagram of a flow chart of an image-based map generation method provided by an embodiment of the present invention;

[0021] Figure 2 A schematic diagram of a target map provided by an embodiment of the present invention;

[0022] Figure 3 A schematic diagram of a flow chart of an image-based map generation method provided by an embodiment of the present invention;

[0023] Figure 4a A schematic diagram of a penetration constraint provided by an embodiment of the present invention;

[0024] Figure 4b A schematic diagram of a smooth constraint provided by an embodiment of the present invention;

[0025] Figure 4c A schematic diagram of a penalty constraint provided by an embodiment of the present invention;

[0026] Figure 5a A schematic diagram of a distance constraint provided by an embodiment of the present invention;

[0027] Figure 5b A schematic diagram of angle constraint provided by an embodiment of the present invention;

[0028] Figure 6 A structural block diagram of an image-based map generation device provided by an embodiment of the present invention;

[0029] Figure 7 A structural block diagram of an image-based map generation device provided by an embodiment of the present invention;

[0030] Figure 8 A structural block diagram of an electronic device provided by an embodiment of the present invention;

[0031] Fig. 9 A structural block diagram of an electronic device provided by an embodiment of the present invention.

[0032] The above drawings have shown clear embodiments of the present invention, which will be described in more detail below. These drawings and text descriptions are not intended to limit the scope of the present invention in any way, but to illustrate the concept of the present invention for those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0033] The following will describe the embodiments of the present invention with reference to the accompanying drawings and preferred embodiments. Those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be understood that the preferred embodiments are only for illustrating the present invention, not for limiting the scope of protection of the present invention.

[0034] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present invention, and thus the drawings only show components related to the present invention rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0035] In the description of the present invention, it should be understood that the terms "first", "second", "third", etc. are only used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence, nor can they be understood as indicating or implying relative importance. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to the specific circumstances. In addition, in the description of the present invention, unless otherwise specified, "plurality" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the objects associated before and after are in an "or" relationship.

[0036] It should be noted that due to space limitations, this specification does not exhaustively list all optional implementations. After reading this specification, those skilled in the art should be able to think that as long as the technical features do not contradict each other, any combination of technical features can constitute an optional implementation. The following is a detailed description of each embodiment.

[0037] In the field of autonomous driving, positioning and mapping are important. Figure 1 It is generally difficult to distinguish. When determining the map, it is always necessary to first accurately locate the vehicle and then obtain a high-precision map of the vehicle's location. However, with the continuous development of the field of autonomous driving, vehicle control on urban roads may lack accurate high-precision maps due to complex environments and geographic information management issues. This results in the vehicle being unable to use high-precision maps as a basis for autonomous driving, and unable to make accurate control instructions, affecting the accuracy and safety of autonomous driving.

[0038] Therefore, in the absence of a high-precision map, it is necessary to bypass the positioning system and directly generate a map with static information and topological information from the image end to guide the vehicle to travel on complex urban roads. The scope of application of the embodiment of the present invention is mainly urban road environment, and can also be applied to environments such as highways. Through a purely visual method for generating an autonomous driving map and topological structure, the production cost of a high-precision map is saved, which has considerable economic benefits, improves the adaptability of autonomous driving vehicles on complex urban roads, and enhances the safety and reliability of vehicle control.

[0039] The present invention provides an image-based map generation method, device, equipment and storage medium, aiming to solve the above technical problems in the prior art.

[0040] The technical solution of the present invention and how the technical solution of the present invention solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present invention will be described below in conjunction with the accompanying drawings.

[0041] Figure 1 FIG. 1 is a flow chart of an image-based map generation method according to an embodiment of the present invention. The method can be executed by an image-based map generation device. Figure 1 As shown, the method comprises the following steps:

[0042] S101. Acquire an environmental image of an environment where a vehicle is located and navigation information of the vehicle; wherein the navigation information includes a driving direction of the vehicle from a starting point to an end point.

[0043] For example, one or more visual sensors may be installed on the vehicle to collect images of the environment in which the vehicle is located. Image collection may be performed in real time or at a fixed time during the driving process of the vehicle to obtain an environmental image of the environment in which the vehicle is located. A panoramic image of the environment in which the vehicle is located may be collected as an environmental image, or images at different angles may be collected and stitched into a complete environmental image.

[0044] The navigation information of the vehicle can also be obtained in real time or at a fixed time. The navigation information can include the driving direction of the vehicle from the starting point to the end point. The navigation information does not include the information of which lane the vehicle is traveling in, but only the general route from the starting point to the end point. That is, the navigation information can represent the driving direction of the vehicle at different positions, but cannot represent which lane is more reasonable for the vehicle, nor can it represent information such as traffic lights at intersections. When the vehicle is driving automatically, the current approximate position can be obtained through GPS (Global Positioning System), and navigation information can be obtained according to the end point position input by the user.

[0045] In this embodiment, obtaining an environmental image of the environment in which the vehicle is located includes: obtaining a local image of the environment in which the vehicle is located in real time through multiple visual sensors installed on the vehicle; wherein the visual sensor includes at least one of a surround view camera and a periscopic camera, and one visual sensor corresponds to one local image; and stitching the local images to obtain an environmental image of the environment in which the vehicle is located.

[0046] Specifically, a plurality of visual sensors may be installed on the vehicle, and each visual sensor may be located at a different position on the vehicle. In the present embodiment, the visual sensor may be a surround view camera and a periscopic camera, etc. Through the plurality of visual sensors on the vehicle, images of the environment in which the vehicle is located may be acquired in real time, and each visual sensor may capture images. Since the visual sensor has its own corresponding acquisition range, the visual sensor can only capture images of the environment within a certain angle, and the image captured by the visual sensor is used as a local image in the environment. That is, one visual sensor corresponds to one local image.

[0047] The local images collected by each visual sensor at the same time are obtained, and these local images are stitched together to obtain a complete environmental image of the environment where the vehicle is located. For example, the local images collected by adjacent visual sensors can be stitched together to obtain a panoramic image of the environment where the vehicle is located.

[0048] The beneficial effect of such a setting is that by acquiring images through multiple cameras, omission of environmental information is avoided and the accuracy of map generation is improved.

[0049] In this embodiment, it also includes: acquiring a current image captured by the visual sensor, and determining the width information of the lane in the current image as the current width information; acquiring the actual width information of the lane; if the difference between the current width information and the actual width information is greater than a preset difference threshold, calibrating the visual sensor to obtain a local image of the environment where the vehicle is located based on the calibrated visual sensor.

[0050] Specifically, each visual sensor is calibrated in advance to reduce the error between the information in the image collected by each visual sensor and the actual information. For each visual sensor, when calibrating, the image collected by the visual sensor is obtained as the current image. The lane width information in the current image is determined. For example, the number of pixels representing the lane width in the current image can be determined. The lane width information in the current image is determined as the current width information, and the current width information is the lane width determined from the image. For example, the lane width has 30 pixels, and the actual distance corresponding to each pixel is 10 centimeters, then the current width information can be determined to be 3 meters. The actual width information of the lane is obtained in advance, and the actual width information is the actual width of the lane. The difference between the current width information and the actual width information is determined. If the difference between the current width information and the actual width information is greater than a preset difference threshold, it is determined that the visual sensor needs to be calibrated, so as to obtain a local image of the environment where the vehicle is located according to the calibrated visual sensor. If the difference between the front width information and the actual width information is less than or equal to the preset difference threshold, it is determined that the visual sensor does not need to be calibrated. In this embodiment, the calibration method of the visual sensor is not specifically limited.

[0051] The preset difference threshold may be a preset fixed value, or one tenth or two tenths of the actual width information. That is, the preset difference threshold may be 0.1 or 0.2 of the actual width information. For example, if the actual width information is 3 meters, the preset difference threshold may be 0.3 meters. If the current width information corresponding to the visual sensor is 3 meters, the difference is 0 meters, and the visual sensor does not need to be calibrated.

[0052] The beneficial effect of this setting is that by calibrating the visual sensor, the error of the visual sensor reprojection can be made within 0.1 to 0.2 of the actual width information, thereby improving the accuracy of obtaining environmental information and further improving the accuracy of map generation.

[0053] In this embodiment, it also includes: acquiring the current image captured by each visual sensor, and determining the position information of the target object in each current image; calibrating the visual sensor according to the position information of the target object in each current image, so as to obtain a local image of the environment where the vehicle is located based on the calibrated visual sensor.

[0054] Specifically, the visual sensors can be calibrated in advance to make the positional relationship of the objects in the images collected by each visual sensor consistent. For example, if two visual sensors both capture the same building, and the building captured by one visual sensor is deformed compared to the building captured by the other visual sensor, then the positional relationship of the objects in the images captured by the two visual sensors is inconsistent. Alternatively, if the number of pixel points of a building captured by one visual sensor is significantly more than the number of pixel points of a building captured by another visual sensor, then the visual sensors also need to be calibrated.

[0055] The current images collected by each visual sensor at the same time are obtained, and the position information of the target object in each current image is determined. The target object is an object in the current image. The current images collected by different visual sensors may include the same object, and the same object may be used as the target object. Generally speaking, the current images collected by two adjacent visual sensors may contain the same object, so every two adjacent visual sensors may be calibrated. The position information refers to the positional relationship of the target object in the current image, for example, it may be the layout of the target object in the current image. The position information of the same target object in different current images is determined, and it is judged whether the position information is consistent. If it is consistent, the visual sensor does not need to be calibrated. If it is inconsistent, the visual sensor needs to be calibrated, so as to obtain a local image of the environment where the vehicle is located according to the calibrated visual sensor. In this embodiment, the external parameters of the visual sensor can be calibrated so that the splicing error of the external parameters of each camera does not exceed 5 pixels.

[0056] The beneficial effect of this setting is that by calibrating the visual sensor, the error in obtaining local images can be reduced and the accuracy of map generation can be improved. The calibration requirement is for the splicing of image results to ensure that map results without obvious human visual differences can be obtained through multiple visual sensors. If there is a certain error tolerance range for the back-end control, the calibration requirements can be appropriately relaxed.

[0057] S102. Determine an initial map of the environment where the vehicle is located according to the environment image; wherein the initial map represents static information in the environment where the vehicle is located.

[0058] For example, by performing image recognition processing on the environment image, information related to the road in the environment where the vehicle is located can be obtained, for example, static information such as the number of lane lines on the road, the width of the lane, whether there is a traffic light in front of the vehicle, and buildings on both sides of the road can be obtained. Based on the obtained static information related to the road, an initial map can be generated. That is, the initial map can represent the static information in the environment where the vehicle is located. For example, lane lines, trees, buildings, traffic lights, etc. can be displayed in the initial map.

[0059] In this embodiment, the image recognition method is not specifically limited. For example, an image recognition model can be pre-trained, and the image recognition model can be used to identify parallel straight lines and dotted lines from the environment image as lane lines. The actual distance corresponding to each pixel point can also be preset to be 10 centimeters, and if it is determined that there are 30 pixels between two parallel straight lines, the lane width can be determined to be 3 meters.

[0060] In this embodiment, determining an initial map of the vehicle's environment based on the environment image includes: determining static information in the vehicle's environment based on the environment image; wherein the static information includes road information and static obstacle information; and generating an initial map of the vehicle's environment based on the static information.

[0061] Specifically, by performing image recognition processing on the environment image, static information in the environment where the vehicle is located can be obtained. The static information may include road information and static obstacle information. The road information may be the layout information of lanes and road markings on the road, and the static obstacle information may be information such as the size and position of static obstacles. Road markings may include lane lines, stop lines, crosswalk lines, etc.

[0062] Based on the static information, an initial map is constructed so that the initial map can represent the actual road layout and the distribution of static obstacles. For example, the static information can be rendered on the initial map. In this embodiment, the construction method of the initial map is not specifically limited.

[0063] The beneficial effect of this setting is that in the absence of high-precision maps, static information in the environment can be determined, and the generation of purely visual autonomous driving maps can be achieved, saving the cost of producing high-precision maps and having considerable economic benefits.

[0064] In this embodiment, static information in the environment where the vehicle is located is determined based on the environmental image, including: performing semantic recognition processing on the environmental image to determine the length of road markings, lane width, and the position and size of static obstacles in the environmental image; determining the length of road markings and lane width as road information, and determining the position and size of static obstacles as static obstacle information.

[0065] Specifically, a semantic recognition model of the image is pre-constructed, and the semantic recognition model is a neural network model. In this embodiment, the model structure of the semantic recognition model is not specifically limited. The environment image is input into the preset semantic recognition model, and the environment image is subjected to feature extraction processing by the semantic recognition model, and the road markings and static obstacles in the environment image are identified according to the extracted feature vectors. Among them, the static obstacles can be traffic lights, trees, buildings, etc.

[0066] According to the identified road markings, the length of the road markings, the type of road markings, and the line type of the road markings can be determined. The road marking types can include lane lines, stop lines, crosswalks, etc., and the line types can include solid lines and dashed lines. According to the road marking type, the lane lines are determined, and the lane width can be determined according to the layout of the lane lines in the environment image. For static obstacles, the position and size of the static obstacles can be determined. The static information includes road information and static obstacle information, wherein the information such as the length of the road markings and the lane width can be determined as road information, and the information such as the position and size of the static obstacles can be determined as static obstacle information, thereby obtaining all static information.

[0067] The beneficial effect of this setting is that the static information includes the layout of lane lines and related information such as static obstacles, which can achieve comprehensive recognition of the vehicle's environment, improve the accuracy of static information extraction, avoid information omissions, and thus improve the accuracy of target map determination.

[0068] S103. Determine a target map of the environment where the vehicle is located according to the initial map and the navigation information; wherein the target map represents static information in the environment where the vehicle is located and information on the vehicle's driving path.

[0069] Exemplarily, after obtaining the initial map, the initial map and navigation information can be combined to determine more accurate information required for vehicle driving, for example, the current specific driving path of the vehicle can be determined. The relevant information of the vehicle's driving path can be determined as topological information. Specifically, the topological information can contain an association structure between different information. In this embodiment, the topological information can include the association relationship between the information of the vehicle's driving path and the information of the road environment. For example, a certain section of the driving path can be associated with the lanes in the road, thereby indicating the driving position, lane changing method, turning method and other driving conditions of the vehicle in the environment. The driving paths at different positions can be associated with different environmental information. The navigation information is combined with the initial map to obtain a target map. That is, static information and topological information can be represented in the target map.

[0070] The topological information can characterize which lane the vehicle can currently drive on, whether the vehicle needs to wait for traffic lights, which lane the vehicle can enter after passing the intersection, etc. The topological information can be determined based on environmental information and navigation information. For example, based on navigation information, it is determined that the vehicle's current driving direction requires a right turn, and based on environmental information, it is determined that the vehicle is at an intersection. The topological information of the vehicle can be obtained as the vehicle turns right from the current lane, passes through the intersection, and enters the lane after the right turn to drive.

[0071] The topological information can be superimposed on the initial map to obtain a target map. For example, if the topological information is the current driving route of the vehicle, the driving route can be displayed on the corresponding lane of the initial map. An arrow can also be displayed on the target map as the current driving direction of the vehicle. Figure 2 is a schematic diagram of the target map. Figure 2 The straight line indicated by an arrow is the current driving path of the vehicle. Figure 2 The dotted line in is the lane line, that is, Figure 2 There are three lanes shown in the figure, and the driving path is located in the middle lane, indicating that the vehicle is driving in the middle lane.

[0072] In an embodiment of the present invention, the approximate driving direction of the vehicle can be determined by acquiring the navigation information of the vehicle. Based on the environmental image, the static information in the environment where the vehicle is located is determined, thereby obtaining an initial image of the environment where the vehicle is located, that is, the initial image only contains static information. Based on the initial map and navigation information, the relevant information of the vehicle's driving path is determined, that is, the topological information is obtained, and the target map is obtained by combining the static information and the topological information. In the absence of a high-precision map, an accurate map containing static information and topological information is provided to the user, thereby assisting the vehicle in automatic driving, improving the accuracy and safety of automatic driving, and enhancing the user's driving experience.

[0073] Figure 3A flowchart of an image-based map generation method provided in an embodiment of the present invention is provided. This embodiment is an optional embodiment based on the above embodiment.

[0074] In this embodiment, a target map of the vehicle's environment is determined based on the initial map and navigation information, including: determining the vehicle's current position through a positioning sensor installed on the vehicle; determining the vehicle's current driving direction based on the navigation information and the current position; and determining the target map of the vehicle's environment based on the initial map and the current driving direction.

[0075] like Figure 3 As shown, the method comprises the following steps:

[0076] S301. Acquire an environmental image of an environment where a vehicle is located and navigation information of the vehicle; wherein the navigation information includes a driving direction of the vehicle from a starting point to an end point.

[0077] Exemplarily, this step may refer to the above-mentioned step S101 and will not be described in detail.

[0078] S302. Determine an initial map of the environment where the vehicle is located according to the environment image; wherein the initial map represents static information in the environment where the vehicle is located.

[0079] Exemplarily, this step may refer to the above-mentioned step S102 and will not be described in detail.

[0080] S303: Determine the current position of the vehicle through a positioning sensor installed on the vehicle.

[0081] For example, a positioning sensor may be installed on the vehicle, and the positioning sensor may be a GPS device. Through the positioning sensor, the current position of the vehicle may be acquired in real time. In this embodiment, the position acquired by the positioning sensor is the geographical location of the vehicle, and it is not possible to determine which lane the vehicle is located in.

[0082] S304: Determine the current driving direction of the vehicle according to the navigation information and the current position.

[0083] Exemplarily, the current position of the vehicle is found from the navigation information, and the current driving direction of the vehicle is determined according to the driving direction of the current position indicated in the navigation information. For example, it can be determined that the vehicle needs to continue to go straight at the current position, and the current driving direction is the direction of going straight.

[0084] S305: Determine a target map of the environment where the vehicle is located according to the initial map and the current driving direction.

[0085] For example, the initial map contains information about lanes and road markings. Based on the information about lanes and road markings, it can be determined which lane the vehicle can be located in, or whether it can change lanes, etc. For example, if the lane line is a solid line, it is determined that the vehicle cannot change lanes. Based on the current driving direction and the static information in the initial map, the driving path of the vehicle can be generated. The direction of the driving path is the current driving direction. The driving path is located on the road in the initial map. The initial map containing the driving path is determined as the target map.

[0086] In this embodiment, the approximate position of the vehicle can be determined by GPS, and then the vehicle can be determined to go straight or turn according to the navigation information, that is, the current driving direction of the vehicle can be determined. Based on the initial map, a target map containing topological information is generated to generate a detailed road map in the absence of a high-precision map, providing a basis for the vehicle's automatic driving and improving the safety and control accuracy of automatic driving.

[0087] In this embodiment, a target map of the vehicle's environment is determined based on the initial map and the current driving direction, including: determining the vehicle's target driving path based on preset constraints according to the initial map and the current driving direction; wherein the preset constraints are used to optimize the driving path; and superimposing the target driving path onto the initial map to obtain a target map of the vehicle's environment.

[0088] Specifically, constraints are pre-set, and the constraints can be used to optimize the driving path to obtain the target driving path currently required by the vehicle, that is, to obtain topological information. Based on the initial map and the current driving direction, multiple driving paths can be generated on the initial map. Based on the preset constraints, the generated multiple driving paths can be optimized to determine the target driving path of the vehicle. For example, multiple driving paths can be screened, and a driving path that meets the constraints can be selected as the target driving path. In this embodiment, the preset constraints are not specifically limited. For example, the preset constraints can be a requirement that the driving path has the least number of lane changes, the shortest driving length, etc.

[0089] The target driving path is superimposed on the initial map as topological information to obtain a target map containing topological information and static information. That is, when the user views the target map, he can see both the surrounding environment displayed on the map and the current driving route of the vehicle.

[0090] The beneficial effect of such a setting is that, based on the preset constraints, the topological information that meets the requirements is determined and the target driving path is obtained, so that the vehicle can drive automatically according to the target driving path, thereby improving the safety of autonomous driving.

[0091] In this embodiment, the target driving path of the vehicle is determined according to the initial map and the current driving direction based on preset constraints, including: determining the current lane of the vehicle based on the environmental image; determining the target lane of the vehicle from the initial map based on the current driving direction and the current lane; and determining the target driving path of the vehicle from the current lane to the target lane based on preset constraints.

[0092] Specifically, by identifying the environment image, the current lane of the vehicle can be determined. For example, the lane where the front of the vehicle is currently located can be determined from the environment image, and the lane where the front of the vehicle is currently located can be used as the current lane. According to the current driving direction and the current lane, the target lane of the vehicle can be determined from the initial map. For example, if the current driving direction is straight ahead, the current lane can be determined as the target lane. Alternatively, if the current driving direction is to turn right, the target lane is the front lane on the right side of the intersection.

[0093] There may be one or more target lanes, and there may be one or more driving paths from the current lane to each target lane. The target driving path for the vehicle to drive from the current lane to the target lane may be determined from the multiple driving paths according to the preset constraints. For example, the path with the shortest driving distance may be found from the multiple driving paths as the target driving path. Alternatively, a path that complies with traffic rules may be found from the multiple driving paths as the target driving path. For example, if there are two driving paths, one of which needs to pass through a solid lane line and the other passes through a dotted lane line, the driving path passing through the dotted lane line is determined as the target driving path.

[0094] The beneficial effect of this setting is to determine which lane the vehicle can drive to as the target lane. Automatically determine the target driving path from the current lane to the target lane, achieve accurate navigation of the vehicle in the absence of high-precision maps, and improve the accuracy of autonomous driving.

[0095] In this embodiment, a target driving path for the vehicle to travel from the current lane to the target lane is determined based on preset constraints, including: determining a candidate driving path for the vehicle to travel from the current lane to the target lane based on a preset first constraint; wherein the preset first constraint represents a constraint on the vehicle's operating rules; and determining a target driving path from the candidate driving paths based on a preset second constraint; wherein the preset second constraint represents a geometric constraint on the driving path.

[0096] Specifically, two constraints can be preset, namely the first constraint and the second constraint. The first constraint is a constraint on the vehicle operation law, so that the finally generated topological information has the rationality of human behavior; the second constraint is a geometric constraint on the driving path, so that the finally generated topological information maintains geometric integrity.

[0097] First, according to the preset first constraint, determine the driving path from the current lane to the target lane as the candidate driving path. That is, the candidate driving path satisfies the first constraint. For example, the first constraint is the traffic rules, and the candidate driving path needs to satisfy the preset traffic rules. After obtaining the candidate driving path, the candidate driving path is screened or optimized according to the preset second constraint to obtain the final target driving path. The second constraint is used to ensure the geometric structure of the topological information. For example, the geometric relationship between the lane line and the path can be ensured. The candidate driving path can be adjusted through the second constraint to adjust the candidate driving path to a geometric relationship parallel to the lane line, so that the vehicle can drive smoothly along the lane line in the lane.

[0098] The beneficial effect of this setting is that by considering the vehicle operation rules and geometric state relationships, two major types of constraints are proposed, which makes the target driving path have good operability and improves the driving experience.

[0099] In this embodiment, based on a preset first constraint condition, a candidate driving path for the vehicle to travel from the current lane to the target lane is determined, including: determining the target position of the vehicle from the target lane; determining the driving path from the current position to the target position as the first driving path; wherein the current position is located in the current lane; smoothing the first driving path to obtain a second driving path; and determining a candidate driving path from the second driving path based on preset traffic driving rules.

[0100] Specifically, after determining the target lane, a position can be determined in the target lane as the target position of the vehicle. For example, the position in the middle of the lane entrance when entering the lane can be used as the target position. Determine all driving paths that can be driven from the current position to the target position as the first driving path. The driving path from the current position to the target position is the driving path from the current lane to the target lane. The first driving path is determined by the current position and the target position, and the penetration constraint on the target driving path is implemented. The penetration constraint means that when the vehicle enters and exits an unstructured location such as an intersection, the topological center line must pass through the two points of entry and exit to ensure the correctness of entry and exit.

[0101] After obtaining the first driving path, the first driving path is smoothed to obtain the second driving path, so as to realize the smooth constraint of the target driving path and improve the riding experience. In this embodiment, the smooth processing process is not specifically limited. The smooth constraint is to consider the basic kinematic law of vehicle operation, that is, geometric smoothness in mathematics or first-order differentiability.

[0102] Traffic driving rules are pre-set as penalty constraints in the first constraint condition. The penalty constraint means that the vehicle cannot perform operations that human drivers should not perform in principle, such as crossing the line, on the generated topology. That is, according to the preset traffic driving rules, a driving path that meets the penalty constraint can be determined from the second driving path as a candidate driving path.

[0103] Figure 4a is a schematic diagram of penetration constraint. Figure 4b is a schematic diagram of smooth constraints, Figure 4c Schematic diagram of penalty constraint. Figure 4a In the figure, A is the current position, B is the target position, and the driving path passes through points A and B. Figure 4b In the example, the driving path has no obvious corners and satisfies the smoothness constraint. Figure 4c In the example, the vehicle crosses the solid line and does not satisfy the penalty constraint.

[0104] The beneficial effect of such a setting is that the first constraint condition may include a penetration constraint, a smoothness constraint and a penalty constraint. Through a variety of constraints, the candidate driving path satisfies the vehicle operation laws, improves the rationality of autonomous driving, and thus improves the riding experience and driving safety.

[0105] In this embodiment, according to the preset second constraint condition, the target driving path is determined from the candidate driving paths, including: determining a third driving path from the candidate driving paths according to the distance between the candidate driving path and the adjacent lane line; determining the target driving path from the third driving path according to the positional relationship between the third driving path and the adjacent lane line.

[0106] Specifically, the geometric characteristics of the topology are analyzed, and geometric constraints are constructed as the second constraint condition. Through the second constraint condition, the finally generated topological structure maintains geometric integrity. The geometric constraints can be determined based on the geometric characteristics of the static elements in the environment and the target driving path, as well as the geometric relationships that need to be maintained after small deformations. The static elements can be static objects such as lane lines on the road. The geometric characteristics and geometric relationships can mean that the lane lines need to be parallel to the target driving path, and the target driving path needs to be in the middle of the lane.

[0107] The second constraint condition may include a distance constraint and an angle constraint. The distance constraint means that the distance between the target driving path and the lane lines on both sides must be within a certain distance range; the angle constraint means that the generated topological structure, that is, the target driving path, and the original lane lines should be consistent in geometric structure to avoid excessive deformation causing damage to the geometric form and causing topological problems.

[0108] For the distance constraint, the distance between the candidate driving path and the adjacent lane lines can be determined, that is, the distance between the candidate driving path and the lane lines on both sides can be determined. A distance range is preset to determine whether the distance between the candidate driving path and the lane lines on both sides is within the preset distance range. If so, the candidate driving path is determined as the third driving path; if not, the candidate driving path is screened out. For example, the distance range is half of the lane width, that is, the distance between the third driving path and the lane lines on both sides should be half of the lane width. For the angle constraint, the target driving path can be determined from the third driving path according to the positional relationship between the third driving path and the adjacent lane lines. A driving path parallel to the lane lines on both sides can be found from the third driving path as the target driving path. If the third driving path is an arc, the third driving path whose tangent to the arc is parallel to the lane lines on both sides can be determined as the target driving path. If multiple target driving paths are determined, the final driving path can be determined by the path length of each target driving path. For example, the target driving path with the shortest length can be determined as the final driving path.

[0109] Figure 5a is a schematic diagram of distance constraint. Figure 5b A schematic diagram of an angle constraint. Figure 5a In the figure, the solid line is the lane line, the dotted line is the third driving path, and the distance between the solid line and the dotted line is half of the lane width. Figure 5b In the example, each lane line is parallel to the driving path.

[0110] The beneficial effect of this setting is that, by imposing geometric constraints on the driving path, it is possible to generate a reasonable target driving path in the absence of a high-precision map, thereby improving the accuracy and safety of autonomous driving.

[0111] In this embodiment, it also includes: obtaining a data set to be trained; wherein the data set to be trained includes an environment image to be trained and a high-precision map corresponding to the environment image to be trained; inputting the environment image to be trained into the initial model to obtain an output map prediction result; wherein the map prediction result represents the map predicted by the initial model based on the environment image to be trained; according to the high-precision map corresponding to the environment image to be trained and the map prediction result, based on a preset first constraint condition and a preset second constraint condition, the initial model is trained to obtain a trained map generation model; wherein the map generation model is used to output a target map of the environment where the vehicle is located based on the environment image of the vehicle and the navigation information of the vehicle.

[0112] Specifically, a map generation model may be pre-trained, and the map generation model may be a neural network model. In this embodiment, the model structure of the map generation model is not specifically limited, for example, a Transformer architecture may be used. When a target map needs to be generated, environmental information and navigation information may be input into the map generation model, and the target map may be output, thereby realizing an end-to-end mapping method with images as input.

[0113] An initial model architecture is pre-constructed as the initial model, and the initial model is trained to obtain a trained map generation model. When performing model training, a data set to be trained can be collected in advance. The data set to be trained may include an environmental image to be trained and a high-precision map corresponding to the environmental image to be trained, that is, an environmental image around the vehicle is collected in advance as the image to be trained, and a high-precision map at the location of the vehicle is obtained according to the location of the vehicle, that is, a high-precision map corresponding to the environmental image to be trained is obtained. When obtaining a high-precision map, the true value of the high-precision map can be intercepted for training based on the positioning results after fusion positioning and Kalman filtering. In this embodiment, for the acquisition of the true value in the Kalman filter of fusion positioning, it can be required that the confidence of the combined inertial navigation sensor in the fixed solution or the final output is above 90%.

[0114] The environment image to be trained and the high-precision map corresponding to the environment image to be trained are input into the initial model. The initial model can obtain the map prediction result according to the environment image to be trained, that is, obtain the predicted map of the current location. According to the preset first constraint and the preset second constraint, the gap between the map prediction result and the high-precision map is determined. During the training process, the map prediction result is gradually approached to the high-precision map. That is, through the high-precision map, the map prediction result is optimized so that the map prediction result satisfies the penetration constraint, smoothness constraint, penalty constraint, distance constraint and angle constraint. Five loss functions can be preset, each loss function can correspond to a constraint, and calculate whether the map prediction result satisfies the corresponding constraint. If the map prediction result satisfies the five constraints, it is determined that the trained map generation model is obtained.

[0115] The beneficial effect of this setting is that by conducting model training, a set of purely visual autonomous driving maps and topological structure generation methods can be constructed, which can directly generate a map structure with static information and topological information from the image end, guide vehicles to drive on complex urban roads, save the cost of producing high-precision maps, and have considerable economic benefits.

[0116] In an embodiment of the present invention, the approximate driving direction of the vehicle can be determined by acquiring the navigation information of the vehicle. Based on the environmental image, the static information in the environment where the vehicle is located is determined, thereby obtaining an initial image of the environment where the vehicle is located, that is, the initial image only contains static information. Based on the initial map and navigation information, the relevant information of the vehicle's driving path is determined, that is, the topological information is obtained, and the target map is obtained by combining the static information and the topological information. In the absence of a high-precision map, an accurate map containing static information and topological information is provided to the user, thereby assisting the vehicle in automatic driving, improving the accuracy and safety of automatic driving, and enhancing the user's driving experience.

[0117] Figure 6 The following is a structural block diagram of an image-based map generation device provided by an embodiment of the present invention. For ease of explanation, only the parts related to the embodiment of the present disclosure are shown. Figure 6 The image-based map generation device 600 includes: an information acquisition module 601, an initial determination module 602 and a target determination module 603.

[0118] The information acquisition module 601 is used to acquire an environmental image of the environment where the vehicle is located and navigation information of the vehicle; wherein the navigation information represents the driving direction of the vehicle from the starting point to the end point;

[0119] An initial determination module 602 is used to determine an initial map of the environment where the vehicle is located according to the environment image; wherein the initial map includes static information in the environment where the vehicle is located;

[0120] The target determination module 603 is used to determine a target map of the environment where the vehicle is located based on the initial map and the navigation information; wherein the target map represents static information in the environment where the vehicle is located and information on the vehicle's driving path.

[0121] Figure 7 A structural block diagram of an image-based map generation device provided in an embodiment of the present disclosure, such as Figure 7 As shown, the image-based map generation device 700 includes an information acquisition module 701 , an initial determination module 702 , and a target determination module 703 , wherein the target determination module 703 includes a position determination unit 7031 , a direction determination unit 7032 , and a target determination unit 7033 .

[0122] The position determination unit 7031 is used to determine the current position of the vehicle through a positioning sensor installed on the vehicle;

[0123] A direction determination unit 7032, configured to determine the current driving direction of the vehicle according to the navigation information and the current position;

[0124] The target determination unit 7033 is used to determine the target map of the environment where the vehicle is located based on the initial map and the current driving direction.

[0125] In one example, the target determination unit 7033 includes:

[0126] A path optimization subunit, configured to determine a target driving path of the vehicle based on the initial map and the current driving direction and based on preset constraints; wherein the preset constraints are used to optimize the driving path;

[0127] The target determination subunit is used to superimpose the target driving path onto the initial map to obtain a target map of the environment where the vehicle is located.

[0128] In one example, the path optimization subunit is specifically used for:

[0129] Determining a current lane of the vehicle according to the environment image;

[0130] Determining a target lane for the vehicle from the initial map according to the current driving direction and the current lane;

[0131] According to the preset constraint conditions, a target driving path of the vehicle from the current lane to the target lane is determined.

[0132] In one example, the path optimization subunit is specifically used for:

[0133] Determining a candidate driving path for the vehicle from the current lane to the target lane according to a preset first constraint condition; wherein the preset first constraint condition represents a constraint on the vehicle's operating law;

[0134] According to a preset second constraint condition, the target driving path is determined from the candidate driving paths; wherein the preset second constraint condition represents a geometric constraint on the driving path.

[0135] In one example, the path optimization subunit is specifically used for:

[0136] determining a target position of the vehicle from the target lane;

[0137] Determine a driving path from the current position to the target position as a first driving path; wherein the current position is located in a current lane;

[0138] Performing smoothing on the first driving path to obtain a second driving path;

[0139] According to preset traffic driving rules, the candidate driving path is determined from the second driving path.

[0140] In one example, the path optimization subunit is specifically used for:

[0141] Determining a third driving path from the candidate driving paths according to the distance between the candidate driving path and an adjacent lane line;

[0142] The target driving path is determined from the third driving path according to the positional relationship between the third driving path and the adjacent lane line.

[0143] In one example, it also includes:

[0144] A data set acquisition module, used to acquire a data set to be trained; wherein the data set to be trained includes an environment image to be trained and a high-precision map corresponding to the environment image to be trained;

[0145] A result prediction module, used for inputting the environment image to be trained into the initial model to obtain a map prediction result; wherein the map prediction result represents the map predicted by the initial model according to the environment image to be trained;

[0146] The model training module is used to train the initial model according to the high-precision map corresponding to the environmental image to be trained and the map prediction result, based on a preset first constraint condition and a preset second constraint condition, to obtain a trained map generation model; wherein the map generation model is used to output a target map of the environment where the vehicle is located based on the environmental image of the environment where the vehicle is located and the navigation information of the vehicle.

[0147] In one example, the initial determination module 702 includes:

[0148] An information determination unit, configured to determine static information in the environment where the vehicle is located according to the environment image; wherein the static information includes road information and static obstacle information;

[0149] An initial determination unit is used to generate an initial map of the environment where the vehicle is located based on the static information.

[0150] In one example, the information determination unit is specifically configured to:

[0151] Performing semantic recognition processing on the environment image to determine the length of road markings, lane width, and the position and size of static obstacles in the environment image;

[0152] The length of the road marking and the lane width are determined as the road information, and the position and size of the static obstacle are determined as the static obstacle information.

[0153] In one example, the information acquisition module 701 includes:

[0154] An image acquisition unit, used to acquire a local image of the environment where the vehicle is located in real time through a plurality of visual sensors installed on the vehicle; wherein the visual sensor includes at least one of a surround view camera and a periscopic camera, and one visual sensor corresponds to one local image;

[0155] The image stitching unit is used to stitch the local images to obtain an environment image of the environment where the vehicle is located.

[0156] In one example, it also includes:

[0157] A width determination module, used to obtain a current image collected by the visual sensor, and determine the lane width information in the current image as current width information;

[0158] An actual determination module, used to obtain actual width information of the lane;

[0159] The first calibration module is used to calibrate the visual sensor if the difference between the current width information and the actual width information is greater than a preset difference threshold, so as to obtain a local image of the environment where the vehicle is located according to the calibrated visual sensor.

[0160] In one example, it also includes:

[0161] An image acquisition module is used to acquire the current images acquired by each visual sensor and determine the position information of the target object in each current image;

[0162] The second calibration module is used to calibrate the visual sensor according to the position information of the target object in each current image, so as to obtain a local image of the environment where the vehicle is located according to the calibrated visual sensor.

[0163] Figure 8 A structural block diagram of an electronic device provided in an embodiment of the present application, such as Figure 8 As shown, the electronic device includes: a memory 81 and a processor 82; the memory 81 is a memory for storing instructions executable by the processor 82.

[0164] The processor 82 is configured to execute the method provided in the above embodiment.

[0165] The electronic device further includes a receiver 83 and a transmitter 84. The receiver 83 is used to receive instructions and data sent by other devices, and the transmitter 84 is used to send instructions and data to external devices.

[0166] Fig. 9It is a block diagram of a terminal device according to an exemplary embodiment. The device may be a mobile phone, a computer, a digital broadcast terminal, a message transceiver, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, or the like.

[0167] Device 900 may include one or more of the following components: a processing component 902 , a memory 904 , a power component 906 , a multimedia component 908 , an audio component 910 , an input / output (I / O) interface 912 , a sensor component 914 , and a communication component 916 .

[0168] The processing component 902 generally controls the overall operation of the device 900, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 902 may include one or more processors 920 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 902 may include one or more modules to facilitate the interaction between the processing component 902 and other components. For example, the processing component 902 may include a multimedia module to facilitate the interaction between the multimedia component 908 and the processing component 902.

[0169] Device 900 may include one or more of the following components: a processing component 902 , a memory 904 , a power component 906 , a multimedia component 908 , an audio component 910 , an input / output (I / O) interface 912 , a sensor component 914 , and a communication component 916 .

[0170] The memory 904 is configured to store various types of data to support operations on the device 900. Examples of such data include instructions for any application or method operating on the device 900, contact data, phone book data, messages, pictures, videos, etc. The memory 904 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0171] The power supply component 906 provides power to the various components of the device 900. The power supply component 906 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the device 900.

[0172] The multimedia component 908 includes a screen that provides an output interface between the device 900 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 908 includes a front camera and / or a rear camera. When the device 900 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0173] The audio component 910 is configured to output and / or input audio signals. For example, the audio component 910 includes a microphone (MIC), and when the device 900 is in an operating mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 904 or sent via the communication component 916. In some embodiments, the audio component 910 also includes a speaker for outputting audio signals.

[0174] I / O interface 912 provides an interface between processing component 902 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.

[0175] The sensor assembly 914 includes one or more sensors for providing various aspects of status assessment for the device 900. For example, the sensor assembly 914 can detect the open / closed state of the device 900, the relative positioning of components, such as the display and keypad of the device 900, and the sensor assembly 914 can also detect the position change of the device 900 or a component of the device 900, the presence or absence of user contact with the device 900, the orientation or acceleration / deceleration of the device 900, and the temperature change of the device 900. The sensor assembly 914 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 914 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 914 may also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0176] The communication component 916 is configured to facilitate wired or wireless communication between the device 900 and other devices. The device 900 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 916 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 916 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0177] In an exemplary embodiment, the apparatus 900 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above method.

[0178] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 904 including instructions, and the instructions can be executed by the processor 920 of the device 900 to perform the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0179] A non-transitory computer-readable storage medium, when instructions in the storage medium are executed by a processor of a terminal device, enables the terminal device to execute the above-mentioned image-based map generation method of the terminal device.

[0180] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present application. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the following claims.

[0181] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A method for generating a map based on an image, characterized in that: include: Acquire an environmental image of the environment in which the vehicle is located and navigation information of the vehicle; wherein the navigation information includes the driving direction of the vehicle from a starting point to an end point; Determine an initial map of the environment in which the vehicle is located according to the environment image; wherein the initial map represents static information in the environment in which the vehicle is located; A target map of the environment where the vehicle is located is determined according to the initial map and the navigation information; wherein the target map represents static information in the environment where the vehicle is located and information on the vehicle's driving path.

2. The method according to claim 1, characterized in that Determining a target map of the environment where the vehicle is located according to the initial map and the navigation information includes: Determining the current position of the vehicle by means of a positioning sensor installed on the vehicle; Determining a current driving direction of the vehicle according to the navigation information and the current position; A target map of the environment where the vehicle is located is determined according to the initial map and the current driving direction.

3. The method according to claim 2, characterized in that Determining a target map of the environment where the vehicle is located according to the initial map and the current driving direction, including: Determining a target driving path of the vehicle based on the initial map and the current driving direction and based on preset constraints; wherein the preset constraints are used to optimize the driving path; The target driving path is superimposed on the initial map to obtain a target map of the environment where the vehicle is located.

4. The method according to claim 3, characterized in that Determining a target driving path of the vehicle according to the initial map and the current driving direction based on preset constraints includes: Determining a current lane of the vehicle according to the environment image; Determining a target lane for the vehicle from the initial map according to the current driving direction and the current lane; According to the preset constraint conditions, a target driving path of the vehicle from the current lane to the target lane is determined.

5. The method according to claim 4, characterized in that Determining a target driving path for the vehicle to travel from the current lane to the target lane according to the preset constraint conditions includes: Determining a candidate driving path for the vehicle from the current lane to the target lane according to a preset first constraint condition; wherein the preset first constraint condition represents a constraint on the vehicle's operating law; According to a preset second constraint condition, the target driving path is determined from the candidate driving paths; wherein the preset second constraint condition represents a geometric constraint on the driving path.

6. The method according to claim 5, characterized in that Determining a candidate driving path for the vehicle from the current lane to the target lane according to a preset first constraint condition includes: determining a target position of the vehicle from the target lane; Determine a driving path from the current position to the target position as a first driving path; wherein the current position is located in a current lane; Performing smoothing on the first driving path to obtain a second driving path; According to preset traffic driving rules, the candidate driving path is determined from the second driving path.

7. The method according to claim 5, characterized in that Determining the target driving path from the candidate driving paths according to the preset second constraint condition includes: Determining a third driving path from the candidate driving paths according to the distance between the candidate driving path and an adjacent lane line; The target driving path is determined from the third driving path according to the positional relationship between the third driving path and the adjacent lane line.

8. The method according to claim 7, characterized in that Also includes: Acquire a data set to be trained; wherein the data set to be trained includes an environment image to be trained and a high-precision map corresponding to the environment image to be trained; Inputting the environment image to be trained into the initial model to obtain a map prediction result; wherein the map prediction result represents the map predicted by the initial model based on the environment image to be trained; According to the high-precision map corresponding to the environmental image to be trained and the map prediction result, based on a preset first constraint condition and a preset second constraint condition, the initial model is trained to obtain a trained map generation model; wherein the map generation model is used to output a target map of the environment where the vehicle is located based on the environmental image of the environment where the vehicle is located and the navigation information of the vehicle.

9. The method according to claim 1, characterized in that: Determining an initial map of the environment where the vehicle is located according to the environment image includes: Determine static information in the environment where the vehicle is located according to the environment image; wherein the static information includes road information and static obstacle information; An initial map of the environment where the vehicle is located is generated based on the static information.

10. The method according to claim 9, characterized in that Determining static information in the environment where the vehicle is located according to the environment image includes: Performing semantic recognition processing on the environment image to determine the length of road markings, lane width, and the position and size of static obstacles in the environment image; The length of the road marking and the lane width are determined as the road information, and the position and size of the static obstacle are determined as the static obstacle information.

11. The method according to claim 1, characterized in that: Obtain an environmental image of the vehicle's environment, including: Acquire a local image of the environment where the vehicle is located in real time through multiple visual sensors installed on the vehicle; wherein the visual sensor includes at least one of a surround view camera and a periscopic camera, and one visual sensor corresponds to one local image; The local images are stitched together to obtain an environmental image of the environment where the vehicle is located.

12. The method according to claim 11, characterized in that Also includes: Acquire a current image captured by the visual sensor, and determine lane width information in the current image as current width information; Obtaining actual width information of the lane; If the difference between the current width information and the actual width information is greater than a preset difference threshold, the visual sensor is calibrated to obtain a local image of the environment where the vehicle is located according to the calibrated visual sensor.

13. The method according to claim 12, characterized in that Also includes: Acquire the current images collected by each visual sensor, and determine the position information of the target object in each of the current images; The visual sensor is calibrated according to the position information of the target object in each of the current images, so as to obtain a local image of the environment where the vehicle is located according to the calibrated visual sensor.

14. An image-based map generation device, characterized in that: include: An information acquisition module, used to acquire an environmental image of the environment in which the vehicle is located and navigation information of the vehicle; wherein the navigation information includes the driving direction of the vehicle from the starting point to the end point; An initial determination module, used to determine an initial map of the environment where the vehicle is located according to the environment image; wherein the initial map represents static information in the environment where the vehicle is located; The target determination module is used to determine a target map of the environment where the vehicle is located based on the initial map and the navigation information; wherein the target map represents static information in the environment where the vehicle is located and information on the vehicle's driving path.

15. An electronic device, characterized in that: include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the method according to any one of claims 1 to 13.

16. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 13 when executed by a processor.

17. A computer program product, characterized in that The invention comprises a computer program, which implements the method according to any one of claims 1 to 13 when being executed by a processor.

Images