Obstacle information generation method and apparatus, electronic device, and computer readable medium

Abstract

Embodiments of the present disclosure disclose an obstacle information generation method and device, electronic equipment and a computer readable medium. A specific embodiment of the method comprises: acquiring a current time road image; performing obstacle detection processing on the current time road image to generate an obstacle vehicle body detection frame edge equation set, an obstacle image detection vertex coordinate set, an obstacle relative angle value, an obstacle detection label and an obstacle position coordinate; generating a detection frame feature point coordinate based on the obstacle vehicle body detection frame edge equation set; and generating obstacle information based on the obstacle feature point coordinate, the obstacle vehicle body detection frame edge equation set, the obstacle image detection vertex coordinate set, the obstacle relative angle value, the obstacle detection label and the obstacle position coordinate. The embodiment can improve the efficiency of the generated obstacle information.

Description

Technical Field

[0001] The embodiments disclosed herein relate to the field of computer technology, and more specifically to obstacle information generation methods, apparatus, electronic devices, and computer-readable media. Background Technology

[0002] Obstacle information generation methods are techniques for determining obstacle information in images. Currently, when generating obstacle information (e.g., if the obstacle is another vehicle, the obstacle information can be distance information and speed information of other vehicles), the common approach is as follows: first, the distorted road image captured by a wide-angle camera is processed to remove distortion; then, the obstacle information is identified from the distorted road image.

[0003] However, the inventors discovered that when generating obstacle information using the above method, the following technical problems often arise:

[0004] First, real-time distortion correction of road images requires a lot of computing resources, which reduces the efficiency of generating obstacle information.

[0005] Second, even if the distortion correction process does not affect the efficiency of obstacle information generation, when there is a certain angle between the obstacle vehicle and the vehicle, the minimum bounding rectangle of the obstacle vehicle detected from the distorted road image is similar to a trapezoid. If this detection box is directly used to generate obstacle information, it is easy to introduce a large error, which will increase the error of the generated obstacle information and thus reduce the accuracy of the generated obstacle information.

[0006] The information disclosed in this background section is only intended to enhance the understanding of the background of the inventive concept, and therefore may contain information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0007] The summary portion of this disclosure is intended to provide a brief overview of the concepts, which will be described in detail in the detailed description portion. This summary portion is not intended to identify key or essential features of the claimed technical solutions, nor is it intended to limit the scope of the claimed technical solutions.

[0008] Some embodiments of this disclosure provide obstacle information generation methods, apparatuses, electronic devices, and computer-readable media to address one or more of the technical problems mentioned in the background section above.

[0009] In a first aspect, some embodiments of this disclosure provide a method for generating obstacle information. The method includes: acquiring a road image at a current time, wherein the road image at the current time is a distorted image; performing obstacle detection processing on the road image at the current time to generate an obstacle vehicle detection box edge equation set, an obstacle image detection vertex coordinate set, obstacle relative angle values, obstacle detection identifiers, and obstacle position coordinates; generating detection box feature point coordinates based on the obstacle vehicle detection box edge equation set; and generating obstacle information based on the obstacle feature point coordinates, the obstacle vehicle detection box edge equation set, the obstacle image detection vertex coordinate set, the obstacle relative angle values, the obstacle detection identifiers, and the obstacle position coordinates.

[0010] Secondly, some embodiments of this disclosure provide an obstacle information generation apparatus, which includes: an acquisition unit configured to acquire a road image at a current time, wherein the road image at the current time is a distorted image; a detection processing unit configured to perform obstacle detection processing on the road image at the current time to generate an obstacle vehicle body detection frame edge line equation set, an obstacle image detection vertex coordinate set, an obstacle relative angle value, an obstacle detection identifier, and obstacle position coordinates; a first generation unit configured to generate detection frame feature point coordinates based on the obstacle vehicle body detection frame edge line equation set; and a second generation unit configured to generate obstacle information based on the obstacle feature point coordinates, the obstacle vehicle body detection frame edge line equation set, the obstacle image detection vertex coordinate set, the obstacle relative angle value, the obstacle detection identifier, and the obstacle position coordinates.

[0011] Thirdly, some embodiments of this disclosure provide an electronic device, including: one or more processors; and a storage device having one or more programs stored thereon, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation of the first aspect above.

[0012] Fourthly, some embodiments of this disclosure provide a computer-readable medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the method described in any of the implementations of the first aspect above.

[0013] The above embodiments of this disclosure have the following beneficial effects: the obstacle information generation method of some embodiments of this disclosure can improve the efficiency of generating obstacle information. Specifically, the reason for the reduced efficiency of generating obstacle information is that real-time distortion correction processing of road images requires a lot of computing resources, thus leading to a decrease in the efficiency of generating obstacle information. Based on this, the obstacle information generation method of some embodiments of this disclosure firstly acquires a road image at the current moment. The road image at the current moment is a distorted image. Secondly, obstacle detection processing is performed on the road image at the current moment to generate an obstacle vehicle body detection box edge equation set, an obstacle image detection vertex coordinate set, obstacle relative angle values, obstacle detection identifiers, and obstacle position coordinates. Here, considering that distortion correction processing of distorted images requires a lot of computing resources, distortion correction processing is not performed, and obstacle detection is performed directly to reduce the consumption of computing resources. Next, based on the obstacle vehicle body detection box edge equation set, the feature point coordinates of the detection box are generated. By generating the coordinates of the feature points in the detection box, errors in obstacle information detected from road images without distortion correction can be mitigated to some extent. Finally, based on the aforementioned obstacle feature point coordinates, the aforementioned obstacle vehicle body detection box edge equations, the aforementioned obstacle image detection vertex coordinates, the aforementioned obstacle relative angle values, the aforementioned obstacle detection markers, and the aforementioned obstacle position coordinates, obstacle information is generated. Thus, obstacle information can be generated without image distortion correction. This reduces computational resource consumption and improves the efficiency of obstacle information generation. Attached Figure Description

[0014] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and elements are not necessarily drawn to scale.

[0015] Figure 1 This is a flowchart of some embodiments of the obstacle information generation method according to the present disclosure;

[0016] Figure 2 These are schematic diagrams illustrating the structure of some embodiments of the obstacle information generation apparatus according to this disclosure;

[0017] Figure 3 This is a schematic diagram of the structure of an electronic device suitable for implementing some embodiments of the present disclosure. Detailed Implementation

[0018] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0019] It should also be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings. Unless otherwise specified, the embodiments and features described in this disclosure can be combined with each other.

[0020] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0021] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0022] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0023] This disclosure will now be described in detail with reference to the accompanying drawings and embodiments.

[0024] Figure 1 A flow 100 of some embodiments of an obstacle information generation method according to the present disclosure is shown. The obstacle information generation method includes the following steps:

[0025] Step 101: Obtain the road image at the current moment.

[0026] In some embodiments, the entity executing the obstacle information generation method can acquire the current road image via wired or wireless means. The current road image can be a distorted image. Alternatively, the current road image can be an image captured by the vehicle-mounted camera in the current frame.

[0027] It should be noted that the aforementioned wireless connection methods may include, but are not limited to, 3G / 4G connection, WiFi connection, Bluetooth connection, WiMAX connection, Zigbee connection, UWB (ultra wideband) connection, and other currently known or future wireless connection methods.

[0028] Step 102: Perform obstacle detection processing on the road image at the current moment to generate the obstacle vehicle body detection box edge line equation set, obstacle image detection vertex coordinate set, obstacle relative angle value, obstacle detection label and obstacle position coordinate.

[0029] In some embodiments, the execution entity can perform obstacle detection processing on the road image at the current moment to generate a set of obstacle vehicle body detection bounding box equations, a set of obstacle image detection vertex coordinates, obstacle relative angle values, obstacle detection identifiers, and obstacle position coordinates. Here, obstacle detection can be performed on the road image at the current moment using a preset obstacle detection algorithm. The obstacle vehicle body detection bounding box equations in the obstacle vehicle body detection bounding box equation set can be the equation of each edge on the detected obstacle vehicle body's three-dimensional minimum bounding box in the vehicle's coordinate system. The value range of the obstacle vehicle body detection bounding box equations can correspond to the length of each edge. The obstacle image detection vertex coordinates in the obstacle image detection vertex coordinate set can be the vertex coordinates of the minimum bounding rectangle of the obstacle's front or rear in the detected image coordinate system. The obstacle relative angle value can be the angle between the obstacle's travel direction and the vehicle's travel direction. The obstacle detection identifier can be used to characterize the type of obstacle vehicle. The obstacle position coordinates can be three-dimensional coordinates in the vehicle's coordinate system.

[0030] As an example, obstacle detection algorithms may include, but are not limited to, at least one of the following: G-CRF (Gauss-conditional random field) model, DenseCRF (Fully-connected Conditional Random Fields) model, MRF (MRF-Markov Random Field) model, etc.

[0031] In some optional implementations of certain embodiments, the execution entity performs obstacle detection processing on the road image at the current moment to generate a set of obstacle vehicle detection box boundary equations, a set of obstacle image detection vertex coordinates, obstacle relative angle values, obstacle detection identifiers, and obstacle position coordinates. This may include the following steps:

[0032] The first step involves performing obstacle feature recognition on the current road image to generate obstacle vehicle detection boxes, obstacle image detection boxes, obstacle relative angle values, obstacle detection markers, and obstacle position coordinates. Obstacle feature recognition can be performed using the obstacle detection algorithm described above. The obstacle vehicle detection box can be the minimum bounding box of the obstacle vehicle's three-dimensional shape. The obstacle image detection box can be the minimum bounding rectangle of the obstacle's front or rear.

[0033] The second step is to determine the equations of each edge line in the obstacle vehicle detection frame, thus obtaining a set of obstacle vehicle detection frame edge line equations. Specifically, the coordinates of each edge on the obstacle vehicle detection frame can be used to determine the corresponding obstacle vehicle detection frame edge line equation.

[0034] The third step is to determine the coordinate values ​​of each vertex on the obstacle image detection box to obtain the obstacle image detection vertex coordinate group.

[0035] Step 103: Generate the coordinates of the feature points of the detection box based on the equation set of the obstacle vehicle body detection box edge line.

[0036] In some embodiments, the execution entity may generate the coordinates of the feature points of the detection box based on the equation set of the obstacle vehicle body detection box edge lines.

[0037] In some optional implementations of certain embodiments, the execution entity generates the coordinates of the feature points of the detection box based on the aforementioned set of equations for the obstacle vehicle body detection box boundary lines, which may include the following steps:

[0038] The first step is to select the obstacle vehicle body detection frame edge equations that satisfy the preset position conditions from the above set of obstacle vehicle body detection frame edge equations, thus obtaining the target edge equation. The preset position conditions can be the obstacle vehicle body detection frame edge equations of the farthest left or right edge within the corresponding obstacle vehicle body's three-dimensional minimum bounding box.

[0039] The second step is to generate the coordinates of the feature points of the detection box based on the aforementioned target edge equation. First, the coordinates of the upper and lower endpoints of the target edge equation can be determined. Second, the sum of one-third of the upper endpoint coordinates and two-thirds of the lower endpoint coordinates can be used to determine the coordinates of the feature points of the detection box.

[0040] In practice, since the far end of the obstacle vehicle is generally closer to the center of the image, and the far end is generally the front of the vehicle on the same lane, the intersection of the front of the vehicle and the edge of the three-dimensional minimum bounding box of the obstacle vehicle is generally located at two-thirds of the edge.

[0041] Step 104: Generate obstacle information based on the obstacle feature point coordinates, the obstacle vehicle body detection box edge line equation set, the obstacle image detection vertex coordinate set, the obstacle relative angle value, the obstacle detection identifier, and the obstacle position coordinates.

[0042] In some embodiments, the execution entity can generate obstacle information based on the obstacle feature point coordinates, the obstacle vehicle body detection box boundary equations, the obstacle image detection vertex coordinates, the obstacle relative angle values, the obstacle detection identifiers, and the obstacle position coordinates. Here, when generating obstacle information, it is assumed that the viewing direction of the vehicle-mounted (forward-looking) camera is horizontal or approximately horizontal to the ground, that is, the viewing direction is approximately perpendicular to the front and rear of the obstacle.

[0043] In some optional implementations of certain embodiments, the execution entity generates obstacle information based on the obstacle feature point coordinates, the obstacle vehicle body detection box edge line equations, the obstacle image detection vertex coordinates, the obstacle relative angle values, the obstacle detection identifiers, and the obstacle position coordinates. This may include the following steps:

[0044] In response to the determination that the abscissa values ​​of the vertex coordinates of each obstacle detection bounding box in the aforementioned obstacle image detection vertex coordinate set are all greater than the abscissa values ​​of the preset camera optical center imaging point coordinates, obstacle information is generated based on the aforementioned obstacle feature point coordinates, the aforementioned obstacle vehicle body detection box edge line equation set, the aforementioned obstacle image detection vertex coordinate set, the aforementioned obstacle relative angle values, the aforementioned obstacle detection identifiers, the aforementioned obstacle position coordinates, and preset observation information. The fact that the abscissa values ​​of the vertex coordinates of each obstacle image detection vertex in the aforementioned obstacle image detection vertex coordinate set are all greater than the preset abscissa values ​​of the camera optical center imaging point coordinates indicates that the obstacle detection box is located in the right half of the road image coordinate system. The camera optical center imaging point coordinates can be the coordinates of the optical center of the vehicle-mounted camera in the road image coordinate system.

[0045] Optionally, the aforementioned implementing entity may also include the following steps:

[0046] The first step is to select obstacle size information from a pre-defined obstacle data table that matches the obstacle detection identifier. This obstacle size information can include the obstacle's height and length. Secondly, the obstacle data table can include various types of standard obstacles and their corresponding size information. The obstacle detection identifier can be used to uniquely identify a type of obstacle in the obstacle data table, thus corresponding to the obstacle size information.

[0047] The second step involves selecting the obstacle image detection vertex coordinates located at the lower left and upper right corners from the aforementioned obstacle image detection vertex coordinate set, which will be used as the first vertex coordinates and the second vertex coordinates. Specifically, the obstacle image detection vertex coordinates located at the lower left corner of the obstacle image detection box can be selected as the first vertex coordinates. Similarly, the obstacle image detection vertex coordinates located at the upper right corner can be selected as the second vertex coordinates.

[0048] The third step involves generating obstacle information based on the obstacle feature point coordinates, the obstacle vehicle detection box boundary equations, the obstacle image detection vertex coordinates, the obstacle relative angle value, the obstacle detection identifier, the obstacle position coordinates, the obstacle height value, the obstacle length value, the first vertex coordinates, and the second vertex coordinates, in response to the determination that the first and second vertex coordinates satisfy a preset vertex condition. The preset vertex position condition can be: the ordinate value of the first vertex coordinate is greater than or equal to the ordinate value of the second vertex coordinate, and the ordinate value of the second vertex coordinate is greater than or equal to the ordinate value of the camera's optical center imaging point coordinate. The first and second vertex coordinates satisfying the preset vertex condition indicates that the entire obstacle image detection box is below the ordinate value of the camera's optical center imaging point coordinate. In practice, when the third preset position condition is met, the border vertex coordinates with minimal image distortion impact can be the obstacle detection border vertex coordinates located at the lower left and upper right corners of the border. However, considering that the minimum bounding rectangle of the obstacle vehicle detected from the distorted road image is similar to a trapezoid, the target obstacle angle value and the obstacle relative coordinate value are generated using the following formula as obstacle information:

[0049] .

[0050] in, This indicates the angle value of the target obstacle. Represents coordinates. This represents the vehicle coordinate system. This represents the optimized relative coordinates of the obstacle within the vehicle's coordinate system. This represents the objective function that minimizes the relative angle value of the obstacle and the coordinates of the corresponding first vertex position in the vehicle coordinate system. , , This represents the preset constraint weights, which can generally be set to 1. This represents the reprojection error of the first vertex coordinates from the image coordinate system to the vehicle coordinate system. This represents the projection error of the obstacle length value. This represents the projection error of the obstacle width value. This means projecting points in the vehicle coordinate system onto the image coordinate system of the road image without distortion correction. This represents the coordinates of the first vertex mentioned above. This represents the coordinates of the second vertex mentioned above. This represents the covariance matrix of the pre-defined observation error for the lower left corner position coordinates corresponding to the first vertex coordinates. This represents the Mahalanobis distance. This indicates the length value of the aforementioned obstacle. This represents the width value of the aforementioned obstacle. This indicates taking the first element (i.e., the x-coordinate value) of the vector within the parentheses. This represents the relative angle value of the aforementioned obstacles. This represents the coordinates of the feature points of the aforementioned obstacles. This represents the covariance matrix of the pre-defined lateral observation error for the coordinate positions of feature points of obstacles. This represents the covariance matrix of the preset lateral observation error for the upper right corner position coordinates corresponding to the second vertex coordinates.

[0051] The above formula and related content, as an inventive point of this disclosure, solve the second technical problem mentioned in the background: "Even if the distortion correction process does not affect the efficiency of obstacle information generation, when there is a certain angle between the obstacle vehicle and the vehicle, the minimum bounding rectangle of the obstacle vehicle detected from the distorted road image is similar to a trapezoid. If this detection box is directly used to generate obstacle information, it is easy to introduce a large error, leading to an increase in the error of the generated obstacle information, and thus reducing the accuracy of the generated obstacle information." The factors that reduce the accuracy of generated obstacle information are often as follows: Even if the distortion correction process does not affect the efficiency of obstacle information generation, when there is a certain angle between the obstacle vehicle and the vehicle, the minimum bounding rectangle of the obstacle vehicle detected from the distorted road image is similar to a trapezoid. If this detection box is directly used to generate obstacle information, it is easy to introduce a large error, leading to an increase in the error of the generated obstacle information. If the above factors are solved, the accuracy of the generated obstacle information can be improved. To achieve this effect, firstly, by introducing preset position conditions, the target edge equation that meets these conditions can be selected. This is then used to generate the feature point coordinates of the detection box. Next, it is determined that the abscissa value of each obstacle detection bounding box vertex in the aforementioned obstacle image detection vertex coordinate group is greater than the abscissa value of the preset camera optical center imaging point coordinate, in order to further refine the position of the obstacle image detection box. This facilitates the selection of suitable obstacle detection vertex coordinates for this position, reducing errors in features within the distorted image. Then, obstacle size information and various covariance matrices are introduced to further eliminate errors in the distorted image data within the aforementioned formula. This improves the accuracy of the generated target obstacle angle values ​​and relative obstacle coordinate values, thereby enhancing the accuracy of the generated obstacle information.

[0052] Optionally, the aforementioned execution entity may also send the obstacle information to a display terminal for display.

[0053] The above embodiments of this disclosure have the following beneficial effects: the obstacle information generation method of some embodiments of this disclosure can improve the efficiency of generating obstacle information. Specifically, the reason for the reduced efficiency of generating obstacle information is that real-time distortion correction processing of road images requires a lot of computing resources, thus leading to a decrease in the efficiency of generating obstacle information. Based on this, the obstacle information generation method of some embodiments of this disclosure firstly acquires a road image at the current moment. The road image at the current moment is a distorted image. Secondly, obstacle detection processing is performed on the road image at the current moment to generate an obstacle vehicle body detection box edge equation set, an obstacle image detection vertex coordinate set, obstacle relative angle values, obstacle detection identifiers, and obstacle position coordinates. Here, considering that distortion correction processing of distorted images requires a lot of computing resources, distortion correction processing is not performed, and obstacle detection is performed directly to reduce the consumption of computing resources. Next, based on the obstacle vehicle body detection box edge equation set, the feature point coordinates of the detection box are generated. By generating the coordinates of the feature points in the detection box, errors in obstacle information detected from road images without distortion correction can be mitigated to some extent. Finally, based on the aforementioned obstacle feature point coordinates, the aforementioned obstacle vehicle body detection box edge equations, the aforementioned obstacle image detection vertex coordinates, the aforementioned obstacle relative angle values, the aforementioned obstacle detection markers, and the aforementioned obstacle position coordinates, obstacle information is generated. Thus, obstacle information can be generated without image distortion correction. This reduces computational resource consumption and improves the efficiency of obstacle information generation.

[0054] Further reference Figure 2 As an implementation of the methods shown in the above figures, this disclosure provides some embodiments of an obstacle information generation device, which are similar to... Figure 1 Corresponding to the method embodiments shown, the device can be specifically applied to various electronic devices.

[0055] like Figure 2As shown, an obstacle information generation apparatus 200 in some embodiments includes: an acquisition unit 201, a detection processing unit 202, a first generation unit 203, and a second generation unit 204. The acquisition unit 201 is configured to acquire a road image at a current time, wherein the road image at the current time is a distorted image; the detection processing unit 202 is configured to perform obstacle detection processing on the road image at the current time to generate an obstacle vehicle body detection frame edge equation set, an obstacle image detection vertex coordinate set, obstacle relative angle values, obstacle detection identifiers, and obstacle position coordinates; the first generation unit 203 is configured to generate detection frame feature point coordinates based on the obstacle vehicle body detection frame edge equation set; the second generation unit 204 is configured to generate obstacle information based on the obstacle feature point coordinates, the obstacle vehicle body detection frame edge equation set, the obstacle image detection vertex coordinate set, the obstacle relative angle values, the obstacle detection identifiers, and the obstacle position coordinates.

[0056] It is understandable that the units described in the device 200 are related to the reference. Figure 1 The steps in the described method correspond to each other. Therefore, the operations, features, and beneficial effects described above for the method also apply to the device 200 and the units contained therein, and will not be repeated here.

[0057] The following is for reference. Figure 3 It shows a schematic diagram of the structure of an electronic device 300 suitable for implementing some embodiments of the present disclosure. Figure 3 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments of this disclosure.

[0058] like Figure 3 As shown, the electronic device 300 may include a processing unit 301 (e.g., a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 302 or a program loaded from a storage device 308 into a random access memory (RAM) 303. The RAM 303 also stores various programs and data required for the operation of the electronic device 300. The processing unit 301, ROM 302, and RAM 303 are interconnected via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0059] Typically, the following devices can be connected to I / O interface 305: input devices 306 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 307 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 308 including, for example, magnetic tapes, hard disks, etc.; and communication devices 309. Communication device 309 allows electronic device 300 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 3 An electronic device 300 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively. Figure 3 Each box shown can represent a device or multiple devices as needed.

[0060] In particular, according to some embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, some embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication device 309, or installed from storage device 308, or installed from ROM 302. When the computer program is executed by processing device 301, it performs the functions defined in the methods of some embodiments of this disclosure.

[0061] It should be noted that, in some embodiments of this disclosure, the computer-readable medium described above may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium may be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In some embodiments of this disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In some embodiments of this disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0062] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol) and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0063] The aforementioned computer-readable medium may be included in the aforementioned device; or it may exist independently and not assembled into the electronic device. The aforementioned computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to: acquire a road image at a current time, wherein the road image at the current time is a distorted image; perform obstacle detection processing on the road image at the current time to generate an obstacle vehicle detection frame edge equation set, an obstacle image detection vertex coordinate set, obstacle relative angle values, obstacle detection identifiers, and obstacle position coordinates; generate detection frame feature point coordinates based on the obstacle vehicle detection frame edge equation set; and generate obstacle information based on the obstacle feature point coordinates, the obstacle vehicle detection frame edge equation set, the obstacle image detection vertex coordinate set, the obstacle relative angle values, the obstacle detection identifiers, and the obstacle position coordinates.

[0064] Computer program code for performing operations of some embodiments of this disclosure can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0065] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0066] The units described in some embodiments of this disclosure can be implemented in software or hardware. The described units can also be housed in a processor; for example, a processor may be described as including an acquisition unit, a detection processing unit, a first generation unit, and a second generation unit. The names of these units do not necessarily limit the specific unit; for example, the acquisition unit may also be described as "a unit that acquires a road image at the current moment."

[0067] The functions described above in this document can be performed at least in part by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), and so on.

[0068] The above description is merely a selection of preferred embodiments of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in the embodiments of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described inventive concept. For example, technical solutions formed by substituting the above-described features with (but not limited to) technical features with similar functions disclosed in the embodiments of this disclosure.

Claims

1. A method for generating obstacle information, comprising: Obtain the road image at the current moment, wherein the road image at the current moment is a distorted image; The road image at the current moment is processed for obstacle detection to generate a set of obstacle vehicle detection box edge equations, a set of obstacle image detection vertex coordinates, obstacle relative angle values, obstacle detection markers, and obstacle position coordinates; Based on the equation set of the obstacle vehicle body detection box edge line, the coordinates of the detection box feature points are generated; Obstacle information is generated based on the coordinates of the feature points of the detection box, the equations of the obstacle vehicle body detection box edges, the coordinates of the detected vertex of the obstacle image, the relative angle value of the obstacle, the obstacle detection identifier, and the obstacle position coordinates; The step of generating obstacle information based on the feature point coordinates of the detection box, the equation set of the obstacle vehicle body detection box edge lines, the coordinate set of the obstacle image detection vertex, the relative angle value of the obstacle, the obstacle detection identifier, and the obstacle position coordinates includes: In response to the determination that the abscissa value of each obstacle detection bounding box vertex coordinate in the obstacle image detection vertex coordinate group is greater than the abscissa value of the preset camera optical center imaging point coordinate, obstacle information is generated based on the detection box feature point coordinates, the obstacle vehicle body detection box edge line equation group, the obstacle image detection vertex coordinate group, the obstacle relative angle value, the obstacle detection identifier, the obstacle position coordinates, and preset observation information; The step of generating obstacle information based on the feature point coordinates of the detection box, the equation set of the obstacle vehicle body detection box edge lines, the coordinate set of the obstacle image detection vertex, the relative angle value of the obstacle, the obstacle detection identifier, the obstacle position coordinates, and preset observation information includes: From a preset obstacle data table, select obstacle size information that matches the obstacle detection identifier, wherein the obstacle size information includes obstacle height value and obstacle length value; Select the coordinates of the obstacle image detection vertices located at the lower left and upper right corners from the obstacle image detection vertex coordinate group, and use them as the first vertex coordinates and the second vertex coordinates; In response to determining that the coordinates of the first vertex and the coordinates of the second vertex satisfy a preset vertex condition, obstacle information is generated based on the coordinates of the feature points of the detection box, the equation set of the edge lines of the obstacle vehicle body detection box, the coordinate set of the vertex of the obstacle image detection, the relative angle value of the obstacle, the obstacle detection identifier, the position coordinates of the obstacle, the height value of the obstacle, the length value of the obstacle, the coordinates of the first vertex, and the coordinates of the second vertex.

2. The method according to claim 1, wherein, The method further includes: The obstacle information is sent to the display terminal for display.

3. The method according to claim 1, wherein, The obstacle detection processing of the road image at the current moment, to generate the obstacle vehicle detection box boundary equation set, the obstacle image detection vertex coordinate set, the obstacle relative angle value, the obstacle detection identifier, and the obstacle position coordinates, includes: Obstacle feature recognition is performed on the road image at the current moment to generate obstacle vehicle detection box, obstacle image detection box, obstacle relative angle value, obstacle detection mark and obstacle position coordinates; Determine the equations of each edge line in the obstacle vehicle body detection frame to obtain the set of edge line equations for the obstacle vehicle body detection frame; The coordinate values ​​of each vertex on the obstacle image detection frame are determined to obtain the obstacle image detection vertex coordinate group.

4. The method according to claim 1, wherein, The step of generating the coordinates of feature points of the detection box based on the equation set of the obstacle vehicle body detection box edge lines includes: Select the obstacle vehicle body detection frame edge line equations that satisfy the preset position conditions from the obstacle vehicle body detection frame edge line equation set to obtain the target edge line equation; Based on the target edge equation, the coordinates of the feature points of the detection box are generated.

5. An obstacle information generation device, comprising: The acquisition unit is configured to acquire a road image at the current time, wherein the road image at the current time is a distorted image; The detection processing unit is configured to perform obstacle detection processing on the road image at the current time to generate a set of obstacle vehicle body detection box edge equations, a set of obstacle image detection vertex coordinates, obstacle relative angle values, obstacle detection identifiers, and obstacle position coordinates; The first generation unit is configured to generate the coordinates of feature points of the detection box based on the equation set of the boundary lines of the obstacle vehicle body detection box. The second generation unit is configured to generate obstacle information based on the coordinates of the feature points of the detection box, the equation set of the edge lines of the obstacle vehicle body detection box, the coordinate set of the vertex of the obstacle image detection, the relative angle value of the obstacle, the obstacle detection identifier, and the obstacle position coordinates. The step of generating obstacle information based on the feature point coordinates of the detection box, the equation set of the obstacle vehicle body detection box edge lines, the coordinate set of the obstacle image detection vertex, the relative angle value of the obstacle, the obstacle detection identifier, and the obstacle position coordinates includes: In response to the determination that the abscissa value of each obstacle detection bounding box vertex coordinate in the obstacle image detection vertex coordinate group is greater than the abscissa value of the preset camera optical center imaging point coordinate, obstacle information is generated based on the detection box feature point coordinates, the obstacle vehicle body detection box edge line equation group, the obstacle image detection vertex coordinate group, the obstacle relative angle value, the obstacle detection identifier, the obstacle position coordinates, and preset observation information; The step of generating obstacle information based on the feature point coordinates of the detection box, the equation set of the obstacle vehicle body detection box edge lines, the coordinate set of the obstacle image detection vertex, the relative angle value of the obstacle, the obstacle detection identifier, the obstacle position coordinates, and preset observation information includes: From a preset obstacle data table, select obstacle size information that matches the obstacle detection identifier, wherein the obstacle size information includes obstacle height value and obstacle length value; Select the coordinates of the obstacle image detection vertices located at the lower left and upper right corners from the obstacle image detection vertex coordinate group, and use them as the first vertex coordinates and the second vertex coordinates; In response to determining that the coordinates of the first vertex and the coordinates of the second vertex satisfy a preset vertex condition, obstacle information is generated based on the coordinates of the feature points of the detection box, the equation set of the edge lines of the obstacle vehicle body detection box, the coordinate set of the vertex of the obstacle image detection, the relative angle value of the obstacle, the obstacle detection identifier, the position coordinates of the obstacle, the height value of the obstacle, the length value of the obstacle, the coordinates of the first vertex, and the coordinates of the second vertex.

6. An electronic device, comprising: One or more processors; Storage device, on which one or more programs are stored, When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-4.

7. A computer-readable medium having a computer program stored thereon, wherein, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-4.

Images