Surround projection methods, devices, computer equipment, and storage media
By constructing and adjusting the surround view projection model, based on environmental and obstacle information, the problems of distortion and misalignment of three-dimensional objects in vehicle surround view technology are solved, improving the three-dimensionality and realism of surround view images and enhancing the accuracy of obstacle judgment.
Patent Information
- Application Number
- CN202310194673.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-02-27
- Publication Date
- 2026-11-10
- Estimated Expiration
- 2043-02-27
AI Technical Summary
Existing vehicle surround view technology suffers from excessive distortion of three-dimensional objects, affecting the three-dimensionality and realism of the image, and misalignment can easily occur in different areas, affecting the accuracy of obstacle judgment.
By acquiring environmental image information, determining scene information and object information of obstacles, an initial surround-view projection model is constructed, and the model is adjusted according to the object information to adapt to the current environment, including adjusting slope data, starting distance and region merging, and using spline curves or cubic curves for smooth transition to ensure that the projection model matches the environment.
It improves the stereoscopic and realistic feel of surround view images, reduces distortion of object projection, and enhances the accuracy of obstacle detection.
Smart Images

Figure CN116188694B_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the field of computer vision technology, and in particular to a surround-view projection method, apparatus, computer equipment, and storage medium. Background Technology
[0002] With the rapid development of computer vision technology and the continuous improvement of mobile hardware processing capabilities, computer vision is increasingly being applied in autonomous driving and is gradually playing an increasingly important role.
[0003] The most widespread application in autonomous driving is the use of images captured by multiple cameras to construct a surround-view panoramic image for driving assistance.
[0004] However, traditional in-vehicle surround view technology suffers from the following problems: While 2D stitched surround view images can provide timely feedback to the driver about the vehicle's surroundings, they suffer from a small field of view and excessive distortion. The driver cannot immediately perceive the vehicle's actual location within the environment. In particular, 2D surround view images without environmental information fusion and lighting processing cannot guarantee image consistency, resulting in a poor viewing experience. Furthermore, while bowl-shaped 3D surround view images can generally present the vehicle's actual driving conditions, these models are typically fixed. Due to limitations in projection principles (curved surface projection) and parallax, abrupt transition areas can easily appear at the boundaries of three-dimensional objects, and significant misalignment can occur at distances.
[0005] In addition, current vehicle surround view technology suffers from excessive distortion of three-dimensional objects in its surround view images, which affects the three-dimensionality and realism of the object projections in the images, and misalignment is easily formed between various areas, which affects the accuracy of the user's judgment of obstacles. Summary of the Invention
[0006] Therefore, it is necessary to provide a panoramic projection method, device, computer equipment, and storage medium to address the above-mentioned technical problems, thereby improving the distortion effect of stereoscopic objects in panoramic images and ensuring the stereoscopic and realistic effect of object projection imaging.
[0007] In a first aspect, this disclosure provides a panoramic projection method. The method includes:
[0008] Acquire environmental image information of the target object, and determine the scene information corresponding to the environmental image information and the object information of the obstacles in the environmental image information;
[0009] At least an initial surround projection model should be created based on the scene information;
[0010] The initial surround projection model is adjusted based on the object information to obtain the target projection model;
[0011] The environmental image information is projected onto the target projection model to obtain a surround view image.
[0012] In one embodiment, adjusting the initial surround projection model based on the object information includes:
[0013] Based on the distance and orientation information from the obstacle to the target object in the object information, adjust the slope data of the spatial part in the initial surround view projection model and the starting distance in the initial surround view projection model.
[0014] In one embodiment, the initial surround projection model includes multiple regions, and the curves on the curved surfaces corresponding to each region are smooth and continuous. The regions are obtained by dividing the initial surround projection model according to region division information. The region division information includes: sensor detection information and / or object range information in the object information.
[0015] In one embodiment, adjusting the initial surround projection model based on the object information further includes:
[0016] Based on the object information, multiple regions of the initial surround view projection model are merged, and the slope data of the spatial part and the starting distance in the initial surround view projection model are adjusted according to the object information.
[0017] In one embodiment, adjusting the slope data of the spatial portion of the initial surround projection model and the initial distance in the initial surround projection model based on the distance and orientation information in the object information includes:
[0018] In response to the distance information being within a preset first range, the slope data and starting distance of the spatial portion on the side corresponding to the obstacle object in the initial surround projection model are adjusted according to the orientation information.
[0019] When the distance information is less than a preset first threshold, the slope data and starting distance of the spatial portion in the initial surround projection model are adjusted to preset slope data and preset starting distance.
[0020] In one embodiment, merging multiple regions of the initial surround projection model based on the object information includes:
[0021] In response to the fact that there are no obstacles in adjacent areas of the plurality of areas, the adjacent areas without obstacles are merged to obtain a first area;
[0022] In response to the presence of the obstacle object in an adjacent region among multiple regions, the adjacent regions containing the obstacle object are merged to obtain a second region;
[0023] The first region and the second region are different regions.
[0024] In one embodiment, merging multiple regions of the initial surround projection model based on the object information includes:
[0025] In response to the absence of the obstacle object in adjacent areas of the plurality of areas, the regions in the plurality of areas where the obstacle object exists are determined;
[0026] In response to the fact that the number of regions with gaps between the regions containing the obstacle is less than a preset threshold, the regions containing the obstacle and the gaps are merged.
[0027] In one embodiment, creating an initial surround-view projection model based at least on the scene information includes:
[0028] At least the model type and parameter information of the initial surround projection model should be determined based on the scene information;
[0029] An initial surround projection model is created based on the model type and parameter information of the initial surround projection model.
[0030] In one embodiment, the method further includes: during the adjustment of the initial surround projection model, using spline curves or cubic curves to gradually adjust the spatial portion and bottom plane portion of the initial surround projection model, and adjusting the shape of the bottom plane portion of the initial surround projection model within a preset range.
[0031] Secondly, this disclosure also provides a surround-view projection device. The device includes:
[0032] The data acquisition module is used to acquire environmental image information of the target object, and determine the scene information corresponding to the environmental image information and the object information of the obstacle objects in the environmental image information;
[0033] The model creation module is used to create an initial surround-view projection model based at least on the scene information;
[0034] The model adjustment module is used to adjust the initial surround projection model according to the object information to obtain the target projection model;
[0035] The projection module is used to project the environmental image information onto the target projection model to obtain a surround view image.
[0036] Thirdly, this disclosure also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of any of the above-described method embodiments.
[0037] Fourthly, this disclosure also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the steps of any of the above-described method embodiments.
[0038] Fifthly, this disclosure also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the steps of any of the above-described method embodiments.
[0039] In the above embodiments, by acquiring environmental image information and determining the scene and object information corresponding to the environmental image information, it is possible to determine the object information corresponding to the obstacle objects in the current scene that needs to be projected. An initial surround-view projection model can be constructed based on the scene information. The constructed initial surround-view projection model can conform to the current environmental scene, and subsequent adjustments are based on this initial surround-view projection model to ensure adaptation to the current scene information and improve the accuracy of environmental image projection. Furthermore, adjusting the initial surround-view projection model according to the object information of the obstacle objects can improve the distortion effect of three-dimensional objects in the environmental image information during projection. After projecting the environmental image information onto the target projection model, the resulting surround-view image can enhance the three-dimensionality and realism of the object projection imaging. Attached Figure Description
[0040] To more clearly illustrate the technical solutions in the specific embodiments of this disclosure or the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.
[0041] Figure 1 This is a schematic diagram of the application environment of the surround projection method in one embodiment;
[0042] Figure 2 This is a flowchart illustrating a surround projection method in one embodiment;
[0043] Figure 3 This is a flowchart illustrating step S206 in one embodiment;
[0044] Figure 4 This is a schematic diagram of an embodiment before and after adjusting the initial surround projection model;
[0045] Figure 5 This is a flowchart illustrating step S302 in one embodiment;
[0046] Figure 6 This is a flowchart illustrating step S304 in one embodiment;
[0047] Figure 7 This is a top view schematic diagram of each region in the initial surround projection model of one embodiment;
[0048] Figure 8 This is a flowchart illustrating step S204 in one embodiment;
[0049] Figure 9 This is a schematic block diagram of the surround projection device in one embodiment;
[0050] Figure 10 This is a schematic diagram of the internal structure of a computer device in one embodiment. Detailed Implementation
[0051] To make the objectives, technical solutions, and advantages of this disclosure clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this disclosure.
[0052] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings herein are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, apparatus, product, or device that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.
[0053] In this article, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.
[0054] This disclosure provides a surround-view projection method that can be applied to, for example... Figure 1In the application environment shown, terminal 102 communicates with sensor 104 via a network. Terminal 102 acquires environmental image information of the target object acquired by sensor 104. Terminal 102 determines the scene information corresponding to the environmental image information and the object information of obstacles in the environmental image information. Terminal 102 creates an initial surround-view projection model based at least on the scene information. Terminal 102 adjusts the initial surround-view projection model according to the object information to obtain a target projection model. Terminal 102 projects the environmental image information onto the target projection model to obtain a surround-view image. Terminal 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, and IoT devices; IoT devices can be smart vehicle devices, etc. Sensor 106 can include cameras, ultrasonic sensors, lidar, etc. The camera can be a surround-view camera composed of multiple wide-angle lenses.
[0055] In one embodiment, such as Figure 2 As shown, a surround projection method is provided, which is applied to... Figure 1 Taking terminal 102 as an example, the explanation includes the following steps:
[0056] S202, acquire environmental image information of the target object, and determine the scene information corresponding to the environmental image information and the object information of the obstacle objects in the environmental image information.
[0057] The environmental image information can be information within the sensor's identifiable surroundings. For example, if the sensor is installed inside a store or station, and the target object is the store or station, the identified environmental image information will be information from inside the store. Similarly, if the sensor is installed around a vehicle, and the target object is the vehicle, the identified environmental image information will be information from the vehicle's surroundings. The specific environmental image information will vary depending on the sensor's location. Scene information can be an open environment or a scene with many obstacles. When the target object is a store or station, obstacles can be shelves, racks, or merchandise shelves; when the target object is a vehicle, obstacles can be surrounding walls, trash cans, or other vehicles. Object information typically includes the distance of obstacles from the sensor, the size, volume, and shape of the obstacles. Environmental image information can be obtained at least through a surround-view camera. For example, a surround-view camera, LiDAR, and ultrasonic sensors can be used together to obtain environmental image information, or a surround-view camera can be used alone. Surround-view projection typically involves projecting the obtained panoramic image (e.g., a 360° image) onto a model to obtain a panoramic image.
[0058] Specifically, current environmental image information can be obtained through sensors installed in different target objects. The terminal acquires the current environmental image information obtained by the sensors. Based on the obstacles present in the environmental image information, the scene information corresponding to the current environmental image information is determined. Based on the obstacles in the environmental image information, the object information corresponding to the obstacle is determined.
[0059] In some exemplary embodiments, taking the example of a sensor installed on a vehicle and acquiring environmental image information around the vehicle, if the current environmental image information contains multiple objects, such as obstacles around the vehicle, and the number of obstacles is greater than a preset threshold (e.g., 3), then the current scene information can be determined to be a scene with many obstacles. If the number of obstacles is less than the preset threshold, then the current scene information around the vehicle can be determined to be an open scene. It is understood that the above is only for illustrative purposes.
[0060] S204, create an initial surround projection model based at least on the scene information.
[0061] The initial panoramic projection model can typically be a model capable of receiving projected images, such as a bowl-shaped model or a boat-shaped model.
[0062] Specifically, different types of projection models are created based on scene information identified under different conditions, and various parameter data corresponding to the projection models are determined in order to create an initial surround-view projection model.
[0063] S206, Adjust the initial surround projection model according to the object information to obtain the target projection model.
[0064] The target projection model typically conforms to the initial surround projection model based on scene and object information.
[0065] Specifically, various parameters in the initial surround projection model can be adjusted based on the object information. For example, the slope and curvature of the solid surface of the initial surround projection model can be adjusted. After adjusting the initial surround projection model, the target projection model can be obtained based on the adjusted initial surround projection model.
[0066] In some exemplary embodiments, for example, when there is a large obstacle, such as a wall, on one side of the vehicle, the slope of the curved surface on the side corresponding to the large obstacle in the initial surround projection model can be adjusted, and the curved surface can be moved closer to the vehicle. After adjustment, the target projection model can be obtained.
[0067] S208, the environmental image information is projected onto the target projection model to obtain a surround view image.
[0068] In some embodiments of this disclosure, projection can typically be a method of projecting image information onto a model.
[0069] Specifically, the above-mentioned target projection model that conforms to scene information and object information has been obtained. Therefore, the environmental image information obtained by the surround-view camera can be projected into the target projection model, and the surround-view image can be obtained based on the projected target projection model.
[0070] In some exemplary embodiments, the processes of adjusting the initial surround view projection model and projecting environmental image information onto the target projection model can both be run on the terminal's GPU to ensure the real-time performance of the surround view images.
[0071] In the aforementioned surround-view projection method, by acquiring environmental image information and determining the corresponding scene and object information, the object information corresponding to obstacles in the current scene to be projected can be determined. An initial surround-view projection model can be constructed based on the scene information. This initial model conforms to the current environmental scene, and subsequent adjustments are based on this model, ensuring adaptation to the current scene information and improving the accuracy of environmental image projection. Furthermore, adjusting the initial surround-view projection model based on the object information of obstacles can improve the distortion effect of three-dimensional objects in the environmental image information during projection. After projecting the environmental image information onto the target projection model, the resulting surround-view image enhances the stereoscopic and realistic feel of the projected object.
[0072] In one embodiment, the initial surround projection model includes multiple regions, and the curves on the curved surfaces corresponding to each region are smooth and continuous. The regions are obtained by dividing the initial surround projection model according to region division information. The region division information includes: sensor detection information and / or object range information in the object information.
[0073] The sensor's detection information typically includes its resolution, such as the minimum sensing angle and / or sensing distance, which are equivalent to the units of sensor data, such as distance accuracy in meters or centimeters, and angular resolution in degrees of 1° or 10°. Object range information typically includes the area occupied by the obstacle and the distance and angle between the obstacle and the sensor.
[0074] Specifically, the initial surround-view projection model can be divided into multiple regions based on sensor detection information. When dividing into multiple regions, it is necessary to ensure that obstacles are within a single region based on object range information. When dividing the regions, contour curves should be used to ensure smooth transitions between regions.
[0075] In some exemplary embodiments, the initial surround projection model can be divided into 360 / A regions on the top projection plane at an angle A (e.g., 10 degrees), where angle A is the angular resolution of the sensor.
[0076] In this embodiment, by dividing the initial surround projection model into multiple regions, the corresponding environmental image information can be used to project each region during projection, thereby improving the projection effect and avoiding projection misalignment. In addition, the projection effect of obstacles can be improved by ensuring that obstacles are within a region based on the object range information.
[0077] In one embodiment, such as Figure 3 As shown, adjusting the initial surround projection model based on the object information includes:
[0078] S302, based on the distance and orientation information from the obstacle to the target object in the object information, adjust the slope data of the spatial part in the initial surround projection model and the starting distance in the initial surround projection model;
[0079] The distance information typically refers to the distance between the obstacle and the sensor mounting location on the target object. For example, when the target object is a vehicle, the distance information could be the information between the obstacle and the vehicle. If the target object is a store or site, the distance information could be the position between the obstacle and the sensor on the target object. The corresponding distance information varies depending on the target object. Those skilled in the art can determine the corresponding distance information based on different target objects. In some embodiments of this disclosure, the distance information typically refers to the distance between the vehicle and the obstacle. Orientation information could be the position information of the vehicle corresponding to the obstacle. Slope data could be the slope of the spatial portion in the initial surround-view projection model. The starting distance could typically be the radius of the bottom region in the initial surround-view projection model.
[0080] Specifically, such as Figure 4 As shown, the slope and starting distance of the initial surround-view projection model can be dynamically adjusted based on the distance of the obstacle from the current sensor and the obstacle's orientation relative to the current sensor. For example, if the initial surround-view projection model is a bowl-shaped model, the slope of the bowl wall can be adjusted to improve the realism and three-dimensionality of the object. The starting distance can be adjusted by varying it within a certain range; a closer distance to the obstacle results in a smaller starting distance and a steeper slope, while a farther distance results in a larger starting distance and a gentler slope. The starting distance can be the radius of a circle or the radius of an ellipse.
[0081] And / or, S304, merge multiple regions of the initial surround projection model according to the object information, and adjust the slope data of the spatial part and the starting distance in the initial surround projection model after merging multiple regions according to the object information.
[0082] Specifically, after the initial surround-view projection model is divided into multiple regions, the following situation may occur: some regions may have no obstacles. If these regions are projected separately, the projection speed may be reduced. Therefore, regions without obstacles can be merged based on object information, and regions with obstacles can also be merged to improve the projection speed. In addition, to enhance the three-dimensionality and realism of the projected objects, it is also necessary to adjust the slope data of the spatial portion and the starting distance in the initial surround-view projection model after merging the regions. The adjustment method in this embodiment can be found in step S302 above, and will not be repeated here.
[0083] S306, in response to the absence of the obstacle object in the environmental image information, the state of the initial surround projection model remains unchanged and the surfaces of adjacent regions between multiple regions remain continuous and smooth.
[0084] Specifically, when there are no obstacles in the environmental image information, although there will be no distortion of the three-dimensional object during projection, in order to improve the realism of the projected image and avoid abrupt transition areas, the initial state of the panoramic projection model can be kept unchanged, and spline curves can be used to make smooth transitions between regions to avoid abrupt transition areas and large misalignment after projection.
[0085] In this embodiment, the slope data of the initial surround-view projection model can be adjusted based on the distance information in the object information to improve the realism and three-dimensionality of the obstacle projection. Simultaneously, multiple regions in the initial surround-view projection model can be merged based on the object information, which can increase the projection speed. Furthermore, adjusting the merged region initial surround-view projection model based on the object information can also enhance the three-dimensionality and realism of the obstacle projection.
[0086] In one embodiment, such as Figure 5 As shown, adjusting the slope data of the spatial portion in the initial surround projection model based on the distance and orientation information in the object information includes:
[0087] S402, determine the relationship between the distance information and the preset first range and the preset first threshold.
[0088] S404, in response to the distance information being within a preset first range of information, adjust the slope data and starting distance of the spatial portion of the initial surround projection model corresponding to the obstacle object according to the orientation information.
[0089] S406, in response to the distance information being less than a preset first threshold, adjust the slope data and starting distance of the spatial portion in the initial surround projection model to preset slope data and preset starting distance.
[0090] S408, in response to the distance information not being within the preset first range information and being greater than the first threshold, the initial surround projection model is not adjusted.
[0091] The first range and the first threshold can be set by those skilled in the art according to different scenarios, such as the recognition range of the sensor. In some embodiments of this disclosure, specific information on the first threshold and the first range is not limited.
[0092] Specifically, when the distance information is within the first range, the slope data of the spatial portion corresponding to the obstacle in the initial surround-view projection model can be adjusted based on the orientation information. For example, the slope of the corresponding side can be adjusted so that the slope gradually becomes steeper as the distance information decreases, reaching a preset threshold. The starting distance of the bottom plane area in the initial surround-view projection model decreases to another preset threshold. It is understood that the above thresholds are only for illustrative purposes. In actual applications, those skilled in the art can set the thresholds themselves, and no absolute limitation is imposed on the thresholds in this disclosure. When the distance information is less than the preset first threshold, the slope data of the initial surround-view projection model can be adjusted to a preset slope data to ensure that the initial surround-view projection model has a fixed shape and does not change. When the distance information is not within the first preset range and the distance information is greater than the first threshold, the obstacle has little impact on the projection effect. Even if the slope data corresponding to the obstacle in the initial surround-view projection model is not adjusted, it will not affect the projection effect of the final surround-view image. Therefore, the initial surround-view projection model does not need to be adjusted.
[0093] In this embodiment, the initial surround projection model is adjusted using the distance information of the obstacle. Based on the relationship between the distance information and the preset first range and the preset first threshold, different ways of adjusting the initial surround projection model can be used to improve the three-dimensionality and realism of the obstacle when it is projected.
[0094] In one embodiment, such as Figure 6 As shown, merging multiple regions of the initial surround projection model based on the object information includes:
[0095] S502, determine whether the obstacle object exists in an adjacent area among multiple areas.
[0096] S504, in response to the fact that there is no obstacle in an adjacent region of the plurality of regions, the adjacent regions without obstacles are merged to obtain a first region.
[0097] S506, in response to the presence of the obstacle object in adjacent areas among multiple regions, the adjacent areas containing the obstacle object are merged to obtain a second region. The first region and the second region are different regions.
[0098] Alternatively, in S508, in response to the fact that no obstacle exists in adjacent areas of the plurality of areas, the regions in which the obstacle exists are determined.
[0099] S510, determine whether the number of regions with gaps between regions containing obstacles is less than a preset threshold.
[0100] S512, in response to the fact that the number of regions with gaps between the regions where the obstacle exists is less than a preset number threshold, the regions where the obstacle exists and the regions with gaps are merged.
[0101] S514, otherwise, no merging process will be performed.
[0102] Specifically, it can be determined whether there are obstacles in adjacent regions among the multiple regions divided by the initial surround-view projection model. If no obstacles exist, adjacent regions without obstacles can be merged into one region to obtain the first region. Since this region has no obstacles, its shape does not need to be adjusted. If obstacles exist in adjacent regions, they can be merged into one region to obtain the second region. Since the merged region contains obstacles, it needs to be adjusted. If there are no obstacles in adjacent regions, but multiple obstacles exist in the environmental image information, it is necessary to determine the regions with obstacles in all regions. These regions with obstacles can be considered target regions. The number of intervals between each target region can be determined. If this number is less than a preset threshold, the target regions and the intervals can be merged into one region, and the merged region can be adjusted. If this number is greater than the preset threshold, merging is not performed.
[0103] like Figure 7As shown, the circular portion represents the spatial portion of the initial surround-view projection model, and the elliptical portion represents the bottom plane region of the initial surround-view projection model. The radius of the bottom plane region can be the starting distance. The model space can be divided into regions A to H. Regions B, D, and E contain obstacles, so regions A, H, G, and F can be merged into one large region. Regions D and E are adjacent regions and also contain obstacles, so they can be merged. Additionally, B and D are separated only by region C. If the preset threshold of 2 is met, then regions B, C, D, and E can be merged into one large region if the interval between them is less than the preset threshold. It should be understood that the above is only for illustrative purposes.
[0104] In this embodiment, multiple regions in the initial surround projection model are merged based on whether there are obstacles in each region, and adjustments are made after merging the regions with obstacles, which can further enhance the stereoscopic and realistic feel of the projected obstacles.
[0105] In one embodiment, such as Figure 8 As shown, the step of creating an initial surround-view projection model based at least on the scene information includes:
[0106] S602, at least determine the model type and parameter information of the initial surround projection model based on the scene information;
[0107] S604, Create an initial surround projection model based on the model type and parameter information of the initial surround projection model.
[0108] The model type can include: boat-shaped or bowl-shaped, etc. Parameter information can include: the size and shape of the bottom planar region of the model, the slope of the curved surface, etc.
[0109] Specifically, the model type and parameter information of the initial surround projection model can be determined at least based on scene information, or it can be determined based on both scene information and object information. Once the model type and parameter information are determined, the initial surround projection model can be constructed based on this model type and parameter information. Various parameters will be adjusted according to different scene information. Alternatively, some common categories can be selected, and corresponding parameters can be adapted according to different categories. The parameters corresponding to each category of scene information can be consistent.
[0110] In some exemplary embodiments, taking the scene information around the vehicle as an example, a bowl-shaped model can be used when the area around the vehicle is relatively open; a boat-shaped model can be used when there are obstacles on both sides of the vehicle. In addition, when there is a large obstacle such as a wall on one side of the vehicle, the slope data of the curved surface on the side corresponding to the obstacle in the initial surround projection model is adjusted to make the slope steeper and closer to the vehicle.
[0111] In this embodiment, different initial surround projection models can be constructed based on different scene information to ensure that the initial surround projection model conforms to the real scene, thereby ensuring the authenticity of the projection.
[0112] In one embodiment, the method further includes: during the adjustment of the initial surround projection model, using spline curves or cubic curves to adjust the curved surface portion and the bottom plane portion of the initial surround projection model in a gradual manner, and adjusting the shape of the bottom plane portion of the initial surround projection model within a preset range.
[0113] The gradual adjustment method typically involves progressively adjusting the slope information of the initial surround-view projection model to avoid abrupt changes in the projected image. During the adjustment of the initial surround-view projection model, smooth connections between regions must be ensured, such as using spline curves or cubic curves for smooth transitions. The function corresponding to the surface must be second-order differentiable. It should be noted that in some embodiments of this disclosure, "smooth" refers to any curve on the surface of the initial surround-view projection model or the target projection model being smooth, continuous, and second-order differentiable, where the smooth surface can be a spline surface. Furthermore, the shape of the bottom plane portion of the initial surround-view projection model is adjusted within a preset range. For example, if the bottom plane portion is an ellipse or a circle, the circular plane of the bottom plane region can be treated as a special elliptical plane. The ellipse can then be stretched or compressed between a preset first distance and a preset second distance to ensure the accuracy of the distances in the bottom plane region of the projected surround-view image.
[0114] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.
[0115] Based on the same inventive concept, this disclosure also provides a surround projection device for implementing the surround projection method described above. The solution provided by this device is similar to the solution described in the above method; therefore, the specific limitations in one or more surround projection device embodiments provided below can be found in the limitations of the surround projection method described above, and will not be repeated here.
[0116] In one embodiment, such as Figure 9 As shown, a surround-view projection device 700 is provided, including: a data acquisition module 702, a model creation module 704, a model adjustment module 706, and a projection module 708, wherein:
[0117] Data acquisition module 702 is used to acquire environmental image information of the target object and determine the scene information corresponding to the environmental image information and the object information of the obstacle objects in the environmental image information;
[0118] The model creation module 704 is used to create an initial surround projection model based at least on the scene information;
[0119] The model adjustment module 706 is used to adjust the initial surround projection model according to the object information to obtain the target projection model;
[0120] The projection module 708 is used to project the environmental image information onto the target projection model to obtain a surround view image.
[0121] In one embodiment of the device, the model adjustment module 706 includes: a first adjustment module, configured to adjust the slope data of the spatial portion and the starting distance in the initial surround projection model based on the distance information and orientation information from the obstacle to the target object in the object information;
[0122] The initial surround projection model includes multiple regions, and the curves on the surface corresponding to each region are smooth and continuous. The regions are obtained by dividing the initial surround projection model according to region division information. The region division information includes: sensor detection information and / or object range information in the object information.
[0123] And / or, the second adjustment module is used to merge multiple regions of the initial surround projection model according to the object information, and adjust the slope data of the spatial part and the starting distance in the initial surround projection model after merging multiple regions according to the object information.
[0124] In one embodiment of the device, the first adjustment module is further configured to, in response to the distance information being within a preset first range, adjust the slope data and starting distance of the spatial portion of the initial surround-view projection model corresponding to the obstacle object according to the orientation information; and in response to the distance information being less than a preset first threshold, adjust the slope data and starting distance of the spatial portion of the initial surround-view projection model to preset slope data and preset starting distance.
[0125] In one embodiment of the device, the second adjustment module is further configured to, in response to the absence of an obstacle in adjacent areas of the plurality of areas, merge the adjacent areas without an obstacle to obtain a first area; and in response to the presence of an obstacle in adjacent areas of the plurality of areas, merge the adjacent areas with the obstacle to obtain a second area; wherein the first area and the second area are different areas. In response to the absence of an obstacle in adjacent areas of the plurality of areas, determine the areas of the plurality of areas containing the obstacle; and in response to the number of areas spaced apart from the areas containing the obstacle being less than a preset threshold, merge the areas containing the obstacle and the spaced areas.
[0126] In one embodiment of the device, the model creation module 704 includes: a model data determination module, used to determine the model type and parameter information of the initial surround projection model based at least on the scene information.
[0127] The model creation submodule is used to create an initial surround projection model based on the model type and parameter information of the initial surround projection model.
[0128] In one embodiment of the device, the model adjustment module 706 is further configured to adjust the spatial portion and bottom plane portion of the initial surround projection model in a gradual manner using spline curves or cubic curves during the adjustment of the initial surround projection model, and the shape of the bottom plane portion of the initial surround projection model is adjusted within a preset range.
[0129] Each module in the aforementioned surround projection device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.
[0130] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 10As shown, the computer device includes a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements a surround-view projection method. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the computer device casing, or an external keyboard, touchpad, or mouse.
[0131] Those skilled in the art will understand that Figure 10 The structure shown is merely a block diagram of a portion of the structure related to the present disclosure and does not constitute a limitation on the computer device to which the present disclosure is applied. A specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.
[0132] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in any of the above method embodiments.
[0133] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in any of the above method embodiments.
[0134] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in any of the above method embodiments.
[0135] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this disclosure can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this disclosure may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this disclosure may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.
[0136] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0137] The embodiments described above are merely illustrative of several implementations of this disclosure, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent disclosure. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this disclosure, and these all fall within the protection scope of this disclosure. Therefore, the protection scope of this disclosure should be determined by the appended claims.
Claims
1. A panoramic projection method, characterized in that, The method includes: Acquire environmental image information of the target object, and determine the scene information corresponding to the environmental image information and the object information of the obstacles in the environmental image information; At least an initial surround projection model should be created based on the scene information; The initial surround projection model is adjusted based on the object information to obtain the target projection model; The environmental image information is projected onto the target projection model to obtain a surround view image; The initial surround projection model includes multiple regions, and the curves on the curved surfaces corresponding to each region are smooth and continuous. The regions are obtained by dividing the initial surround projection model according to region division information. The region division information includes: sensor detection information and / or object range information in the object information. The sensor detection information includes the resolution of the sensor, and the object range information includes the range occupied by the obstacle object and the distance and angle between the obstacle object and the sensor. The step of adjusting the initial surround projection model based on the object information further includes: Based on the object information, multiple regions of the initial surround view projection model are merged, and the slope data of the spatial part and the starting distance in the initial surround view projection model are adjusted according to the object information.
2. The method according to claim 1, characterized in that, The step of adjusting the initial surround projection model based on the object information includes: Based on the distance and orientation information from the obstacle to the target object in the object information, adjust the slope data of the spatial part in the initial surround projection model and the starting distance in the initial surround projection model.
3. The method according to claim 2, characterized in that, The step of adjusting the slope data of the spatial portion and the starting distance in the initial surround projection model based on the distance and orientation information in the object information includes: In response to the distance information being within a preset first range, the slope data and starting distance of the spatial portion on the side corresponding to the obstacle object in the initial surround projection model are adjusted according to the orientation information. When the distance information is less than a preset first threshold, the slope data and starting distance of the spatial portion in the initial surround projection model are adjusted to preset slope data and preset starting distance.
4. The method according to claim 1, characterized in that, The step of merging multiple regions of the initial surround projection model based on the object information includes: In response to the fact that there are no obstacles in adjacent areas of the plurality of areas, the adjacent areas without obstacles are merged to obtain a first area; In response to the presence of the obstacle object in an adjacent region among multiple regions, the adjacent regions containing the obstacle object are merged to obtain a second region; The first region and the second region are different regions.
5. The method according to claim 1, characterized in that, The step of merging multiple regions of the initial surround projection model based on the object information includes: In response to the absence of the obstacle object in adjacent areas of the plurality of areas, the regions in the plurality of areas where the obstacle object exists are determined; In response to the fact that the number of regions with gaps between the regions containing the obstacle is less than a preset threshold, the regions containing the obstacle and the gaps are merged.
6. The method according to claim 1, characterized in that, The step of creating an initial surround-view projection model based at least on the scene information includes: At least the model type and parameter information of the initial surround projection model should be determined based on the scene information; An initial surround projection model is created based on the model type and parameter information of the initial surround projection model.
7. The method according to any one of claims 1 to 6, characterized in that, The method further includes: during the adjustment of the initial surround projection model, using spline curves or cubic curves to gradually adjust the spatial part and the bottom plane part of the initial surround projection model, and adjusting the shape of the bottom plane part of the initial surround projection model within a preset range.
8. A surround-view projection device, characterized in that, The device includes: The data acquisition module is used to acquire environmental image information of the target object, and determine the scene information corresponding to the environmental image information and the object information of the obstacle objects in the environmental image information; A model creation module is used to create an initial surround-view projection model based on at least the scene information. The initial surround-view projection model includes multiple regions, and the curves on the curved surfaces corresponding to each region are smooth and continuous. The regions are obtained by dividing the initial surround-view projection model according to region division information. The region division information includes: sensor detection information and / or object range information in the object information. The sensor detection information includes the resolution of the sensor, and the object range information includes the range occupied by the obstacle object and the distance and angle between the obstacle object and the sensor. The model adjustment module is used to adjust the initial surround projection model according to the object information to obtain the target projection model; A projection module is used to project the environmental image information onto the target projection model to obtain a surround view image; The model adjustment module includes: The first adjustment module is used to merge multiple regions of the initial surround projection model according to the object information, and adjust the slope data of the spatial part and the starting distance of the initial surround projection model after merging multiple regions according to the object information.
9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.
11. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.
Citation Information
Patent Citations
Adjustment method and device of vehicle-mounted look-around model, electronic equipment and storage medium
CN113689552A
Panoramic image generation method and device and automatic driving equipment
CN115442541A
System and Method for Distortion Correction in Three-Dimensional Environment Visualization
US20140278065A1
