An obstacle warning method, device, medium, and equipment for reversing assistance

Abstract

This application discloses an obstacle warning method, device, medium, and equipment for reversing assistance. The method includes: constructing several target reversing auxiliary lines based on historical reversing images containing target reference objects, and determining the target position of each target reversing auxiliary line in the historical reversing images; performing obstacle recognition on the real-time captured current reversing image to identify several obstacle objects; obtaining the current position of each obstacle object in the current reversing image; and performing warning detection on each obstacle object based on the target position of each target reversing auxiliary line and the current position, so as to provide obstacle warning based on the detection results. This application realizes obstacle warning by utilizing reversing images, that is, only a single monocular camera is needed to capture reversing images to achieve obstacle warning detection, eliminating the need for other detection devices, reducing the use of detection devices, and lowering the cost of obstacle warning.

Description

Technical Field

[0001] This invention relates to the field of vehicle technology, and in particular to a method, apparatus, medium, and equipment for constructing reversing auxiliary lines. Background Technology

[0002] With the continuous improvement of economic level, automobiles, as a means of transportation, have greatly facilitated people's travel and provided convenience for people's lives.

[0003] Existing car reversing assist systems typically provide obstacle warnings to prevent collisions. However, current reversing systems usually employ obstacle detection methods based on binocular vision or a combination of binocular vision and lidar, which involve numerous detection devices and are therefore costly. Summary of the Invention

[0004] In view of this, the present invention provides an obstacle warning method, device medium and equipment for reversing assistance, the main purpose of which is to solve the problem that the current obstacle warning detection involves a large number of detection devices and has a high cost.

[0005] To address the aforementioned problems, this application provides an obstacle warning method for reversing assistance, comprising:

[0006] Based on historical reversing images containing target reference objects, several target reversing auxiliary lines are constructed, and the target position of each target reversing auxiliary line in the historical reversing image is determined.

[0007] Obstacle recognition is performed on the real-time captured image of the vehicle reversing to identify several obstacle objects;

[0008] Obtain the current position of each obstacle object in the current reversing image;

[0009] Based on the target position of each of the reversing auxiliary lines and the current position of each of the objects, a warning detection is performed on each of the obstacles, and an obstacle warning is given based on the detection results.

[0010] Optionally, the step of constructing several target reversing auxiliary lines based on historical reversing images containing target reference objects includes:

[0011] In response to the user's selection operation of the target reference object in the historical reversing image and the selection operation of each pixel region in the historical reversing image, the coordinate data corresponding to each pixel region is obtained;

[0012] Several target reversing auxiliary lines are constructed based on the coordinate data described above.

[0013] Optionally, before constructing several target reversing auxiliary lines based on the coordinate data, the method further includes:

[0014] For each coordinate data, respond to the auxiliary line type selection command respectively, configure the corresponding auxiliary line type for each coordinate data, and obtain the coordinate dataset corresponding to each auxiliary line type;

[0015] The construction of several target reversing auxiliary lines based on the coordinate data includes:

[0016] Based on the coordinate data in the coordinate dataset corresponding to the same auxiliary line type, generate target reversing auxiliary lines of the straight line type corresponding to the auxiliary line type to obtain several target reversing auxiliary lines;

[0017] Alternatively, based on the coordinate data in the coordinate dataset corresponding to the same auxiliary line type, an optimization process can be used to generate target reversing auxiliary lines of the curve type that correspond to the auxiliary line type, so as to obtain several target reversing auxiliary lines.

[0018] Optionally, the auxiliary line type includes any one or more of the following types: left auxiliary line type along the vehicle length direction, right auxiliary line type along the vehicle length direction, first auxiliary line type along the vehicle width direction, second auxiliary line type along the vehicle width direction, and third auxiliary line type along the vehicle width direction; the distance between the target reversing auxiliary line corresponding to the first auxiliary line type, the second auxiliary line type, and the third auxiliary line type and the rear of the vehicle is different.

[0019] Optionally, the step of generating a target reversing auxiliary line of curve type corresponding to the auxiliary line type by using optimized processing of each coordinate data in the coordinate dataset corresponding to the same auxiliary line type includes:

[0020] Based on the coordinate data in the coordinate dataset corresponding to the same auxiliary line type, curve fitting is performed to obtain the current auxiliary line corresponding to the auxiliary line type.

[0021] Based on at least each current auxiliary line, calculate the current error value corresponding to each current auxiliary line;

[0022] Based on the current error value of the current auxiliary line, the corresponding current auxiliary line is optimized until the predetermined optimization stopping condition is met. The current auxiliary line is then used as the target reversing auxiliary line to obtain the target reversing auxiliary line corresponding to each auxiliary line type.

[0023] Optionally, the step of calculating the current error value corresponding to each current auxiliary line, at least based on each current auxiliary line, includes:

[0024] Based on each current auxiliary line and each coordinate data in the coordinate dataset corresponding to each current auxiliary line, calculate the distance value between each coordinate data and the current auxiliary line;

[0025] The current error value of the current auxiliary line is calculated based on each of the aforementioned distance values.

[0026] Optionally, the step of optimizing the corresponding current auxiliary line based on the current error value of the current auxiliary line until a predetermined optimization stopping condition is met, and using the current auxiliary line as the target reversing auxiliary line to obtain the target reversing auxiliary line corresponding to each auxiliary line type, includes:

[0027] Based at least on the current error value of the current auxiliary line and the predetermined error threshold, determine whether the current auxiliary line meets the optimization stopping condition;

[0028] If it is determined that the current auxiliary line does not meet the optimization stopping condition, the coordinate data in the coordinate dataset corresponding to the current auxiliary line are optimized according to the predetermined optimization method to obtain an optimized coordinate dataset, and the current auxiliary line is refitted based on the coordinate data in the optimized coordinate dataset.

[0029] If the current auxiliary line is determined to meet the optimized stopping conditions, the current auxiliary line will be used as the target reversing auxiliary line.

[0030] Optionally, determining whether the current auxiliary line meets the optimization stopping condition based at least on the current error value of the current auxiliary line and a predetermined error threshold includes:

[0031] Compare the current error value with a predetermined error threshold;

[0032] If the current error value is less than or equal to the error threshold, or the number of optimization attempts reaches a predetermined number, it is determined that the optimization stopping condition is met.

[0033] If the current error value is greater than the error threshold and the number of optimization attempts has not reached the predetermined number, it is determined that the optimization stop condition is not met.

[0034] Optionally, the predetermined optimization method includes any one or more of the following methods:

[0035] Based on the coordinate data of the current auxiliary line and the Euclidean distance between each coordinate data, each coordinate data is optimized to obtain an optimized coordinate data set corresponding to the current auxiliary line.

[0036] And / or, based on the coordinate data of the current auxiliary line and the distance between each coordinate data and the current auxiliary line, optimize each coordinate data to obtain an optimized coordinate data set corresponding to the current auxiliary line;

[0037] And / or, based on the method of exchanging horizontal and vertical coordinates, perform coordinate transformation on each coordinate data in the coordinate dataset corresponding to the current auxiliary line to obtain an optimized coordinate dataset corresponding to the current auxiliary line.

[0038] Optionally, the target reference object includes a target calibration field, and the method further includes:

[0039] In the designated parking area, n*n squares are drawn according to the preset side length to pre-arrange the target calibration field.

[0040] Optionally, before performing warning detection on each obstacle object based on the target position of each target reversing guide line and each current position, the method further includes:

[0041] Determine the object category of each obstacle object, and obtain the target warning level corresponding to each obstacle based on the object category of each obstacle object;

[0042] The method of detecting and warning of obstacles based on the target positions of the reversing auxiliary lines and the current positions of the obstacles includes:

[0043] Based on the target position of each target reversing auxiliary line, obtain the warning range corresponding to each target warning level, so as to obtain the warning range corresponding to each obstacle object;

[0044] Based on the warning range corresponding to each obstacle object and the current position of each obstacle object, a warning detection is performed on each obstacle object to determine whether each obstacle object meets the corresponding warning conditions, and an obstacle warning is issued when the warning conditions are met.

[0045] Optionally, the method further includes:

[0046] Based on the shooting time of the current reversing image, obtain the adjacent reversing images that are adjacent to the shooting time;

[0047] Determine the historical position of each obstacle object in the adjacent reversing images;

[0048] Based on the historical and current positions of each obstacle object, determine the motion parameters of each obstacle object;

[0049] Based on the motion parameters of each obstacle, warning information is provided for each obstacle in the current reversing image.

[0050] To address the aforementioned problems, this application provides an obstacle warning device for reversing assistance, comprising:

[0051] The module is used to construct several target reversing auxiliary lines based on historical reversing images containing target reference objects, and to determine the target position of each target reversing auxiliary line in the historical reversing image;

[0052] The recognition module is used to identify obstacles in the real-time captured image of the current reversing vehicle and determine several obstacle objects.

[0053] The first acquisition module is used to acquire the current position of each obstacle object in the current reversing image;

[0054] The early warning module is used to perform early warning detection on each of the obstacle objects based on the target position of each of the target reversing auxiliary lines and the current position of each of the objects, so as to provide obstacle warning based on the detection results.

[0055] To address the aforementioned problems, this application provides a storage medium storing a computer program that, when executed by a processor, implements the steps of the obstacle warning method for reversing assistance described above.

[0056] To address the aforementioned problems, this application provides an electronic device, comprising at least a memory and a processor, wherein the memory stores a computer program, and the processor, when executing the computer program in the memory, implements the steps of the obstacle warning method for reversing assistance described above.

[0057] The obstacle warning method, device, medium, and equipment for reversing assistance described in this application can conveniently and quickly construct reversing auxiliary lines by utilizing historical reversing images containing target reference objects. Subsequently, based on the target position of the reversing auxiliary lines in the reversing image, the position of each obstacle in the reversing image can be detected for obstacle warning. This realizes obstacle warning by using reversing images, that is, only a single monocular camera is needed to capture reversing images to achieve obstacle warning detection, without the need to detect the actual distance between the obstacle and the vehicle. Therefore, no other detection devices are required, reducing the use of detection devices and lowering the cost of obstacle warning.

[0058] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and in order to make the above and other objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description

[0059] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0060] Figure 1 This is a flowchart illustrating an obstacle warning method for reversing assistance according to an embodiment of this application;

[0061] Figure 2 This is a schematic diagram of the target calibration field in an embodiment of this application;

[0062] Figure 3 This is a schematic diagram illustrating the relationship between the target reversing guide line and each warning range in the embodiments of this application;

[0063] Figure 4 This is a schematic diagram of three straight target reversing guide lines in another embodiment of this application;

[0064] Figure 5 This is a schematic diagram of five straight target reversing guide lines in another embodiment of this application;

[0065] Figure 6 This is a flowchart illustrating the optimization process of the target auxiliary line according to another embodiment of this application;

[0066] Figure 7 This is a structural block diagram of an obstacle warning device for reversing assistance according to another embodiment of this application;

[0067] Figure 8 This is a structural block diagram of an electronic device according to another embodiment of this application. Detailed Implementation

[0068] Various embodiments and features of this application are described herein with reference to the accompanying drawings.

[0069] It should be understood that various modifications can be made to the embodiments described herein. Therefore, the above description should not be considered as limiting, but merely as an example of embodiments. Other modifications within the scope and spirit of this application will be apparent to those skilled in the art.

[0070] The accompanying drawings, which are included in and form part of this specification, illustrate embodiments of the present application and, together with the general description of the present application given above and the detailed description of the embodiments given below, serve to explain the principles of the present application.

[0071] These and other features of this application will become apparent from the following description of preferred forms, given as non-limiting examples, with reference to the accompanying drawings.

[0072] It should also be understood that although this application has been described with reference to some specific examples, those skilled in the art can certainly implement many other equivalent forms of this application.

[0073] The above and other aspects, features and advantages of this application will become more apparent when taken in conjunction with the accompanying drawings and in view of the following detailed description.

[0074] Specific embodiments of this application are described thereafter with reference to the accompanying drawings; however, it should be understood that the claimed embodiments are merely examples of this application, which can be implemented in various ways. Well-known and / or repeated functions and structures are not described in detail to avoid unnecessary or redundant details that could obscure the application. Therefore, the specific structural and functional details claimed herein are not intended to be limiting, but merely serve as the basis and representative basis for the claims to teach those skilled in the art to use this application in a variety of substantially any suitable detailed structures.

[0075] This specification may use the phrases “in one embodiment,” “in another embodiment,” “in yet another embodiment,” or “in other embodiments,” all of which may refer to one or more of the same or different embodiments according to this application.

[0076] This application provides an obstacle warning method for reversing assistance, specifically applicable to a reversing assistance system within a vehicle. This system includes at least a camera, a display / touchscreen, and a processor. The method in this embodiment can be specifically applied to the processor within the reversing assistance system. Figure 1 As shown, the method in this embodiment includes the following steps:

[0077] Step S101: Based on the historical reversing image containing the target reference object, construct several target reversing auxiliary lines and determine the target position of each target reversing auxiliary line in the historical reversing image;

[0078] In this step, a camera can be installed at the rear of the vehicle. Before obstacle warning, the camera can capture images of the rear of the vehicle, including the target reference object, thus obtaining historical reversing images. Specifically, based on these historical reversing images, coordinate data can be obtained, and then several target reversing auxiliary lines can be constructed using this coordinate data.

[0079] The target reference object can be a pre-marked / arranged n*n grid, or an obstacle object at a certain distance from the rear of the vehicle. For example, an obstacle 3 meters or 2.5 meters from the rear of the vehicle, etc. These obstacles can be trees, people, vehicles, signs, traffic cones, etc. In this step, by displaying a reversing image containing the reference object, and then selecting the target reference object in the reversing image using a human-computer interaction method, several coordinate data points for generating the target reversing guide line can be obtained conveniently and quickly. This ensures the reasonable and accurate construction of the target reversing guide line based on the coordinate data.

[0080] In this step, by constructing the target reversing guide line, the target position (pixel position) of the target reversing guide line in the reversing image can be determined, which lays the foundation for subsequent real-time warning detection of obstacles in the reversing image based on the position of the target reversing guide line in the reversing image.

[0081] Step S102: Obstacle recognition is performed on the real-time captured current reversing image to identify several obstacle objects;

[0082] In the specific implementation of this step, the YOLOv7 object detection model based on deep learning can be used to identify various obstacle objects in the reversing image. Specifically, the identified obstacle objects can be represented by a bounding box.

[0083] Step S103: Obtain the current position of each obstacle object in the current reversing image;

[0084] In this step, after identifying an obstacle, the pixel position of that obstacle in the reversing image can be further determined, thus obtaining the current position corresponding to the obstacle. For example, when an obstacle is identified, a bounding box can be used to identify it, and then the pixel position of that bounding box in the reversing image can be determined, thereby obtaining the current position of the obstacle in the reversing image.

[0085] Step S104: Based on the target position of each of the target reversing auxiliary lines and the current position of each of the targets, perform early warning detection on each of the obstacle objects, so as to provide obstacle warning based on the detection results.

[0086] In this step, several warning ranges can be determined based on the target positions of the reversing guide lines for each target. Then, based on the current positions of each obstacle, the positional relationship between each obstacle and the warning range is detected to obtain the detection results. When an obstacle is detected to be within the warning range, a warning prompt can be issued, such as outputting a warning prompt message according to a predetermined output method. The output method can be, for example, voice, text, identifiers, sound, etc.

[0087] The obstacle warning method for reversing assistance in this embodiment can easily and quickly construct a target reversing guide line by utilizing historical reversing images containing target reference objects. Subsequently, based on the target position of the target reversing guide line in the reversing image, the position of each obstacle in the reversing image can be detected and obstacle warnings can be performed. This realizes obstacle warning by using reversing images, that is, only a single monocular camera is needed to capture reversing images to achieve obstacle warning detection, without the need to detect the actual distance between the obstacle and the vehicle. Therefore, no other detection devices are required, reducing the use of detection devices and lowering the cost of obstacle warning.

[0088] Based on the above embodiments, another embodiment of this application provides an obstacle warning method for reversing assistance. In this embodiment, the specific obstacle warning process is as follows:

[0089] Step S201: Display historical reversing images containing the target reference object. The historical reversing images are scene images captured by a camera device at the rear of the vehicle after the vehicle has traveled to a predetermined distance from the target reference object.

[0090] The target reference object can be a pre-marked / arranged n*n grid, or an obstacle object at a certain distance from the rear of the vehicle. For example, 10*10 squares can be drawn on the ground of the predetermined parking area with a side length of 1 meter, thus obtaining a grid-like target calibration field, where the side length of the squares can be set and adjusted according to actual needs. Other examples include obstacles 3 meters or 2.5 meters from the rear of the vehicle, where obstacles can be trees, people, vehicles, signs, traffic cones, etc.

[0091] Taking a grid-like target calibration field as an example, n*n squares with preset side lengths can be drawn in a predetermined parking area to pre-arrange a grid-like target calibration field for use as the target reference. For example, 10*10 squares can be drawn on the ground of the predetermined parking area with a side length of 1 meter to obtain a grid-like target calibration field. The side length of the squares can be set and adjusted according to actual needs, such as 0.2 meters, 0.5 meters, 0.8 meters, 1.5 meters, etc. The number of squares can also be adjusted according to actual needs, such as 5*5, 6*8, 10*8, etc.

[0092] Then, the vehicle can be controlled to drive to the edge of the target calibration field, and the bottom of the rear of the vehicle can be kept parallel to the edge line of the squares of the target calibration field, that is, the distance between the rear of the vehicle and the edge line of the squares of the target calibration field can be kept to zero. Thus, the camera at the rear of the vehicle can be used to capture a historical reversing image containing the target reference object, and the historical reversing image can be displayed using a display device / screen / touch screen.

[0093] Step S202: In response to the user's selection operation of the target reference object in the historical reversing image and the pixel region in the historical reversing image, obtain the coordinate data corresponding to each pixel region;

[0094] In this step, since the side length of each square in the grid-like target calibration field is known (e.g., 0.5 meters, 1 meter, etc.), the user can select the corresponding pixel area in the displayed historical reversing image based on the desired distance to the target reversing guide line, thereby obtaining the coordinate data corresponding to the pixel area. The selection operation can be a circle-like selection or a click-like selection, etc.

[0095] That is, for example, regarding such Figure 2The grid-like target calibration field shown has each square having a side length of 1 meter. The user expects to set the target reversing guide line distances as follows: 3 meters from the rear of the vehicle (3 meters perpendicular to the rear of the vehicle) and 5 meters from the rear of the vehicle (5 meters perpendicular to the rear of the vehicle). Based on the location of the rear of the vehicle in the historical reversing image and the grid-like target calibration field in the historical reversing image, the user can determine the boundary area between the 3rd and 4th rows of squares from the rear of the vehicle in the historical reversing image as the pixel area to be selected. Alternatively, the user can determine the area of ​​the vertices of the squares between the 3rd and 4th rows of squares in the historical reversing image as the pixel area to be selected. The user can then select or click on the pixel area to be selected in the displayed historical reversing image on the screen / touchscreen to obtain the coordinate data corresponding to each pixel area.

[0096] For example, regarding such Figure 2 The grid-like target calibration field shown has each square having a side length of 1 meter. The user expects to set the target reversing guide line at the following distances: 0.5 meters from the left side of the vehicle (0.5 meters perpendicular to the left side of the vehicle) and 0.5 meters from the right side of the vehicle (0.5 meters perpendicular to the right side of the vehicle). Based on the position areas of the vehicle's sides in the historical reversing image and the grid-like target calibration field in the historical reversing image, the user can determine the area connecting the center points of the first column of squares on the side of the vehicle in the historical reversing image, which is the pixel area to be selected. The user can then select or click on the pixel area to be selected in the displayed historical reversing image on the screen / touchscreen to obtain the coordinate data corresponding to each pixel area.

[0097] In this step, by pre-arranging a grid-like target calibration field, users can quickly and accurately determine the coordinate data / coordinate points in historical reversing images where the desired actual reversing distance is located, based on the actual size of the squares in the target calibration field and their positions in the target reversing image. This ensures the rapid and accurate drawing of target reversing auxiliary lines that meet user needs based on the coordinate data. It enables dynamic adjustment and modification of the target reversing auxiliary lines, satisfying the personalized setting needs of different users for reversing auxiliary lines, and laying the foundation for subsequent obstacle warning based on the target reversing auxiliary lines.

[0098] Step S203: Construct several target reversing auxiliary lines based on the coordinate data.

[0099] In this step, after obtaining the coordinate data, the system can respond to the selection command for each auxiliary line type, configuring the corresponding auxiliary line type for each coordinate data to obtain a coordinate dataset corresponding to each auxiliary line type. Specifically, after obtaining several coordinate data points through human-computer interaction, auxiliary line types can be configured for each coordinate data point to obtain a coordinate dataset corresponding to each auxiliary line type. For example, the corresponding auxiliary line type can be configured for each coordinate data point on the type configuration page. For instance, the coordinate data points / coordinate points can be numbered according to the order of selection operations, and then the type configuration operation can be performed sequentially on the type configuration page for each numbered coordinate data point. For example, for any target coordinate data point to be configured, several auxiliary line type options (virtual buttons) can be displayed on the type configuration page. Users can then generate selection commands through single-click, double-click, and drag operations. The processor in the reversing assist system can respond to these selection commands to configure the corresponding auxiliary line type for the target coordinate data. Users can then trigger the "Next" button to select the next numbered target coordinate data point for type configuration.

[0100] The auxiliary line types include any one or more of the following types: left auxiliary line type along the vehicle length direction, right auxiliary line type along the vehicle length direction, first auxiliary line type along the vehicle width direction, second auxiliary line type along the vehicle width direction, and third auxiliary line type along the vehicle width direction; the distance between the target reversing auxiliary line and the rear of the vehicle is different for the first auxiliary line type, the second auxiliary line type, and the third auxiliary line type.

[0101] After configuring the auxiliary line type for each coordinate data point, target reversing auxiliary lines of the same type but different line type can be generated based on the coordinate data in the coordinate dataset corresponding to that auxiliary line type, thus obtaining several target reversing auxiliary lines. In other words, coordinate data from the coordinate dataset corresponding to the same auxiliary line type can be obtained. For example, by obtaining the coordinate data from the coordinate dataset corresponding to the left auxiliary line type, and then drawing a straight-line left auxiliary line based on that set of coordinate data, a target reversing auxiliary line is obtained. Similarly, by obtaining the coordinate data from the coordinate dataset corresponding to the first auxiliary line type, and then drawing a straight-line first auxiliary line based on that set of coordinate data, another target reversing auxiliary line is obtained. This method of generating reversing auxiliary lines is fast and efficient because it generates straight-line reversing auxiliary lines.

[0102] Alternatively, after configuring the auxiliary line type for each coordinate data, an optimized processing method can be used to generate a target reversing auxiliary line of the same type but with a curve, based on the coordinate data in the coordinate dataset corresponding to the same auxiliary line type. This yields several target reversing auxiliary lines. Specifically, the coordinate data in the coordinate dataset corresponding to the same auxiliary line type is obtained. For example, the coordinate data in the coordinate dataset corresponding to the left auxiliary line type is obtained, and then an optimized processing method is used to draw a curve-type left auxiliary line, thus obtaining the target reversing auxiliary line. Similarly, the coordinate data in the coordinate dataset corresponding to the first auxiliary line type is obtained, and then an optimized processing method is used to draw a curve-type first auxiliary line, thus obtaining another target reversing auxiliary line. This process can yield several target reversing auxiliary lines. In this auxiliary line generation method, because the camera at the rear of the vehicle may use a wide-angle or ultra-wide-angle lens, distortion and perspective issues may occur. This causes the generated straight-line reversing auxiliary lines to deviate from the actual reversing distance, failing to accurately reflect the mapping relationship between pixel coordinates in the image coordinate system and the true distance. Therefore, by using the coordinate data to generate curve-type target reversing guide lines through optimized processing, the target reversing guide lines can meet the imaging distortion of the optical lens, and the generated curve-type target reversing guide lines can accurately reflect the mapping relationship between pixel coordinates and real distances in the image coordinate system. In other words, the generated target reversing guide lines are more accurate and reliable.

[0103] Step S204: Obstacle recognition is performed on the real-time captured current reversing image to identify several obstacle objects;

[0104] In the specific implementation of this step, the YOLOv7 object detection model based on deep learning can be used to identify various obstacle objects in the reversing image. Specifically, the identified obstacle objects can be represented by a bounding box.

[0105] Step S205: Obtain the current position of each obstacle object in the current reversing image;

[0106] In this step, after identifying an obstacle, the pixel position of that obstacle in the reversing image can be further determined, thus obtaining the current position corresponding to the obstacle. For example, when an obstacle is identified, a bounding box can be used to identify it, and then the pixel position of that bounding box in the reversing image can be determined, thereby obtaining the current position of the obstacle in the reversing image.

[0107] Step S206: Determine the object category of each obstacle object, and obtain the target warning level corresponding to each obstacle object based on the object category of each obstacle object;

[0108] In this step, the YOLOv7 deep learning object detection model can also be used to identify the categories of each obstacle object, thereby obtaining the object category. The object categories include: human body, vehicle, animal, and other categories. "Other categories" refers to objects or plants other than human bodies, vehicles, and animals, such as guardrails, rocks, and trees. In practice, corresponding warning levels can be pre-configured for each object category. For example, a level 1 warning level can be configured for human body objects; a level 1 warning level for vehicle objects; a level 2 warning level for animal objects; and a level 3 warning level for other categories, meaning that other categories only require a lenient warning. Therefore, after obtaining the object category of the obstacle object, its corresponding target warning level can be obtained.

[0109] Step S207: Based on the target position of each target reversing auxiliary line, obtain the warning range corresponding to each target warning level, so as to obtain the warning range corresponding to each obstacle object;

[0110] In the specific implementation of this step, several warning ranges can be pre-defined based on the target positions of each target reversing guide line, and then corresponding warning levels can be configured for each warning range. Taking the target reversing guide line as an example... Figure 3 Taking the five reversing guide lines shown as an example, a first warning range v, a second warning range w, and a third warning range u can be defined. The third warning range u can be determined based on the second warning range w, specifically the area near the rear of the car within the second warning range w. Then, a first warning level can be configured for the first warning range v, a second warning level for the second warning range w, and a third warning level for the third warning range u. The first warning range includes the second warning range; that is, the first warning range v is greater than the second warning range w, and the second warning range w is greater than the third warning range u. Therefore, once the target warning level corresponding to the obstacle object is determined, the warning range corresponding to the obstacle object can be determined based on the mapping relationship between warning levels and warning ranges. In the specific implementation of this embodiment, the category of each obstacle can be set and adjusted according to actual needs, and thus the warning range corresponding to each obstacle category can also be adjusted according to actual needs.

[0111] Step S208: Based on the warning range corresponding to each obstacle object and the current position of each obstacle object, perform warning detection on each obstacle object to determine whether each obstacle object meets the corresponding warning conditions, and issue an obstacle warning when the warning conditions are met.

[0112] In this step, after determining the warning range for each obstacle, the current position of each obstacle can be used to determine whether it falls within the warning range. This determines whether the obstacle meets the warning conditions. If it falls outside the warning range, the warning conditions are met, and an obstacle warning can be issued. Specifically, obstacle warnings can be issued through voice prompts, buzzer alerts, or by marking the warned obstacles on the display interface using predefined icons.

[0113] In this embodiment, different warning messages can be used for different object categories. For example, a buzzer can be used to warn objects of level one (humans or vehicles); voice warnings can be used for objects of level two (animals). The specific warning method can be set and adjusted according to actual needs. In this embodiment, different warning measures are taken for different categories of detected obstacles, broadening the warning range for categories of objects with little impact on safety to reduce false alarms, while making the strictest judgments for objects such as people and vehicles that have a significant impact on driving safety to ensure driving safety.

[0114] In this embodiment, by constructing several target reversing auxiliary lines and determining several warning ranges based on each target reversing auxiliary line, it is possible to accurately detect whether each obstacle object is within the corresponding warning range. When an obstacle object is detected to be within the corresponding warning range, it is determined that the warning condition is met, thereby enabling obstacle warning. This realizes obstacle warning based on reversing images, meaning that only a single monocular camera is needed to capture reversing images to achieve obstacle warning detection. There is no need to detect the actual distance between the obstacle and the vehicle, thus eliminating the need for other detection devices, reducing the use of detection devices, and lowering the cost of obstacle warning.

[0115] In specific implementation, the above embodiments can also acquire adjacent reversing images based on the shooting time of the current reversing image; determine the historical position of each obstacle object in the adjacent reversing images; determine the motion parameters of each obstacle object based on its historical and current positions; and provide warning information to each obstacle object in the current reversing image based on its motion parameters. That is, the movement speed and direction of each obstacle object can be determined based on its historical and current positions, and then warning information can be provided based on the movement speed and direction. For example, an arrow can be used to indicate the direction of movement of the obstacle at the location corresponding to the obstacle object, and movement speed information can be displayed at the location corresponding to the obstacle object.

[0116] Based on the above embodiments, another embodiment of this application provides an obstacle warning method for reversing assistance. In this embodiment, when obtaining the coordinate data for constructing several target reversing assistance lines, the following two methods can be used:

[0117] Method 1: In response to the user's selection operation of the target reference object in the historical reversing image and the pixel region within the historical reversing image, obtain several pixel regions; obtain the coordinate range of each pixel region, and determine the coordinate data corresponding to each pixel region based on the coordinate range of each pixel region.

[0118] In this method, when the target reference object is an object or a grid-like target calibration field, the pixel area selected by the user may or may not contain grid vertices. Therefore, when it does not contain grid vertices, its coordinate range can be determined based on the selected pixel area, and then the center coordinate point in the coordinate range can be determined as the target coordinate point corresponding to the pixel area. This is how coordinate data is obtained, making the determination of coordinate points / coordinate data more reasonable and accurate.

[0119] Method 2: In response to the user's selection operation of the target reference object in the historical reversing image and the pixel region within the historical reversing image, obtain several pixel regions; identify the grid vertices of the target reference object in each pixel region to obtain the coordinate data corresponding to each pixel region.

[0120] In this method, when the target reference object is a grid-shaped target calibration field, the pixel area selected by the user may or may not contain grid vertices. Therefore, when it contains grid vertices, that is, when the pixel area is identified to contain grid vertices, the coordinates of the grid vertices can be further obtained, thereby obtaining the coordinate points corresponding to the pixel area, that is, obtaining the coordinate data corresponding to the pixel area, making the determination of coordinate points / coordinate data more reasonable and accurate.

[0121] In the specific implementation process, after obtaining the pixel region, grid vertex identification can be performed on the target pixel region first. If no grid vertices are identified, the coordinate data corresponding to each pixel region is determined based on the coordinate range of each pixel region. If grid vertices are identified, the coordinate data corresponding to each pixel region is obtained based on the identified grid vertices. By adopting the above method to obtain coordinate data, the acquisition of coordinate data can be made more accurate and reliable, laying the foundation for the subsequent accurate generation of the target reversing auxiliary line based on each coordinate data.

[0122] Based on the above embodiments, another embodiment of this application provides an obstacle warning method for reversing assistance. In this embodiment, the obstacle warning method is constructed and obtained as follows: Figure 4When the target reversing guide line shown contains three straight lines, the construction process of each target reversing guide line is as follows:

[0123] Step 1: Draw n*n squares in the designated parking area according to the preset side length to pre-arrange a grid-like target calibration field for use as a target reference object;

[0124] In this step, the side length of each square can be set to 0.5 meters. In this step, n can be any natural number greater than zero.

[0125] Step 2: Control the vehicle to drive to the edge of the target calibration field, and keep the bottom of the vehicle's rear end parallel to the edge line of the squares in the target calibration field. Then, take a historical reversing image containing the target reference object and display it.

[0126] Step 3: Respond to the user's selection operation of the target reference object in the historical reversing image and the pixel area within the historical reversing image, obtain the coordinate data corresponding to each pixel area.

[0127] In other words, users can select the desired distances of 1 meter, 3 meters, and 5 meters from historical reversing images along the car's central axis. Figure 3 The pixel regions a, b, and c are shown, thus obtaining coordinate data A corresponding to pixel region a, coordinate data B corresponding to pixel region b, and coordinate data C corresponding to pixel region c. Specifically, this yields coordinate data A corresponding to a distance of 1 meter, coordinate data B corresponding to a distance of 3 meters, and coordinate data C corresponding to a distance of 5 meters. Multiple pixel regions a, b, and c can be selected as needed, resulting in several sets of coordinate data A, B, and C.

[0128] Step 4: For each coordinate data, respond to the auxiliary line type selection command and configure the corresponding auxiliary line type for each coordinate data to obtain the coordinate dataset corresponding to each auxiliary line type.

[0129] That is, configure the first auxiliary line type for coordinate data / coordinate point A, configure the second auxiliary line type for coordinate data / coordinate point B, and configure the third auxiliary line type for target coordinate data / coordinate point C.

[0130] Step 5: Based on the coordinate data in the coordinate dataset corresponding to the same auxiliary line type, generate a straight target reversing auxiliary line corresponding to the auxiliary line type.

[0131] That is, based on coordinate data A, a first target reversing auxiliary line corresponding to the first auxiliary line type and along the vehicle width direction is generated; based on coordinate data B, a second target reversing auxiliary line corresponding to the second auxiliary line type and along the vehicle width direction is generated; and based on coordinate data C, a third target reversing auxiliary line corresponding to the third auxiliary line type and along the vehicle width direction is generated, thereby obtaining... Figure 3 The image shows three target reversing guide lines. These three target reversing guide lines can then be displayed, for example, the first target reversing guide line can be displayed in red, the second target reversing guide line in yellow, and the third target reversing guide line in green.

[0132] Based on the above embodiments, another embodiment of this application provides an obstacle warning method for reversing assistance. In this embodiment, the obstacle warning method is constructed and obtained as follows: Figure 4 When the target reversing guide line shown contains five straight lines, the construction process of each target reversing guide line is as follows:

[0133] Step 1: Draw n*n squares in the designated parking area according to the preset side length to pre-arrange a grid-like target calibration field for use as a target reference object;

[0134] In this step, the side length of each square can be set to 0.5 meters. In this step, n can be any natural number greater than zero.

[0135] Step 2: Control the vehicle to drive to the edge of the target calibration field, and keep the bottom of the vehicle's rear end parallel to the grid edge line of the target calibration field. Then, capture and display a historical reversing image containing the target reference object.

[0136] Step 3: Respond to the user's selection operation of the target reference object in the historical reversing image and the pixel area within the historical reversing image, obtain the coordinate data corresponding to each pixel area.

[0137] In other words, based on their desired distances of 1 meter, 3 meters, 5 meters from the rear of the car, and 0.5 meters from the side of the car, users can select the desired distance along the car's length axis in the reversing image. Figure 4The pixel regions d, e, f1, f2, g1, and g2 shown are used to obtain coordinate data D1 corresponding to pixel region d, F2 corresponding to pixel region f2, G1 corresponding to pixel region g1, and G2 corresponding to pixel region g2. This yields coordinate data D corresponding to a distance of 1 meter, E corresponding to a distance of 3 meters, F1 and F2 corresponding to a distance of 5 meters, and F1, F2, G1, and G2 corresponding to a distance of 0.5 meters from the side of the vehicle. Multiple pixel regions can be selected as needed to obtain a number of coordinate data points.

[0138] Step 4: For each coordinate data, respond to the auxiliary line type selection command and configure the corresponding auxiliary line type for each coordinate data to obtain the coordinate dataset corresponding to each auxiliary line type.

[0139] That is, configure the first auxiliary line type for coordinate data D, the second auxiliary line type for coordinate data E, the third auxiliary line type for coordinate data F1 and F2, the right auxiliary line type for coordinate data F1 and G1, and the left auxiliary line type for coordinate data F2 and G2.

[0140] Step 5: Based on the coordinate data in the coordinate dataset corresponding to the same auxiliary line type, generate a straight target reversing auxiliary line corresponding to the auxiliary line type.

[0141] That is, based on coordinate data D, a first target reversing auxiliary line corresponding to the first auxiliary line type and along the vehicle width direction is generated; based on coordinate data E, a second target reversing auxiliary line corresponding to the second auxiliary line type and along the vehicle width direction is generated; based on coordinate data F1 and F2, a third target reversing auxiliary line corresponding to the third auxiliary line type and along the vehicle width direction is generated; based on coordinate data F1 and G1, a right target reversing auxiliary line corresponding to the right auxiliary line type and along the vehicle length direction is generated; and based on coordinate data F2 and G2, a left target reversing auxiliary line corresponding to the left auxiliary line type and along the vehicle length direction is generated. This yields... Figure 5 The diagram shows five target reversing guide lines. These five target reversing guide lines can be displayed, for example, the first target reversing guide line can be displayed in red, the second target reversing guide line in yellow, the third target reversing guide line in green, and the left and right target reversing guide lines in blue.

[0142] Based on the above embodiments, another embodiment of this application provides an obstacle warning method for reversing assistance. In this embodiment, the construction process of each target reversing auxiliary line is as follows when constructing the curved target reversing auxiliary line:

[0143] Step 1: Draw n*n squares in the designated parking area according to the preset side length to pre-arrange a grid-like target calibration field for use as a target reference object;

[0144] In this step, the side length of each square in the grid-like target calibration field can be set to 0.5 meters. In this step, n can be any natural number greater than zero.

[0145] Step 2: Control the vehicle to drive to the edge of the target calibration field, and keep the bottom of the vehicle's rear end parallel to the edge line of the squares in the target calibration field. Then, take a historical reversing image containing the target reference object and display it.

[0146] Step 3: Respond to the user's selection operation based on the target reference object in the historical reversing image and the pixel region within the historical reversing image, obtain the coordinate data corresponding to each pixel region;

[0147] Step 4: For each coordinate data, respond to the auxiliary line type selection command respectively, configure the corresponding auxiliary line type for each coordinate data, and obtain the coordinate dataset corresponding to each auxiliary line type;

[0148] Step 5: Based on the coordinate data in the coordinate dataset corresponding to the same auxiliary line type, perform curve fitting to obtain the current auxiliary line corresponding to the auxiliary line type;

[0149] Step 6: Calculate the current error value corresponding to each current auxiliary line, based at least on each current auxiliary line.

[0150] In this step, the distance between each current auxiliary line and its corresponding coordinate data in the coordinate dataset can be calculated. Then, the current error value of the current auxiliary line can be calculated based on these distance values. For example, the average distance can be used as the current error value of the current auxiliary line; alternatively, the variance or standard deviation of the distances can be calculated and used as the current error value of the current auxiliary line.

[0151] Step 7: Optimize the corresponding current auxiliary line based on the current error value of the current auxiliary line until the predetermined optimization stopping condition is met, and use the current auxiliary line as the target reversing auxiliary line to obtain the target reversing auxiliary line corresponding to each auxiliary line type.

[0152] In this step, specifically, it can be determined whether the current auxiliary line meets the optimization stopping condition based at least on the current error value of the current auxiliary line and a predetermined error threshold. If it is determined that the current auxiliary line does not meet the optimization stopping condition, the coordinate data in the coordinate dataset corresponding to the current auxiliary line are optimized according to a predetermined optimization method to obtain an optimized coordinate dataset. The current auxiliary line is then refitted based on the coordinate data in the optimized coordinate dataset. If it is determined that the current auxiliary line meets the optimization stopping condition, the current auxiliary line is used as the target reversing auxiliary line.

[0153] In the specific implementation process, the current error value can be compared with a predetermined error threshold; if the current error value is less than or equal to the error threshold, or the number of optimizations reaches a predetermined number of times, it is determined that the optimization stopping condition is met; if the current error value is greater than the error threshold and the number of optimizations has not reached the predetermined number of times, it is determined that the optimization stopping condition is not met.

[0154] In this embodiment, the predetermined optimization method includes any one or more of the following:

[0155] Method 1: Based on the Euclidean distance between the coordinate data in the coordinate dataset corresponding to the current auxiliary line, optimize each coordinate data to obtain an optimized coordinate dataset corresponding to the current auxiliary line.

[0156] In this step, for the coordinate dataset corresponding to the current auxiliary line, the Euclidean distance (L2 distance) between each coordinate data in the dataset can be calculated. Based on this Euclidean distance, sparsification is performed on the coordinate data, meaning that only one of the two coordinate data with similar Euclidean distances is retained, thus obtaining an optimized coordinate dataset corresponding to the current auxiliary line. In practice, each Euclidean distance can be compared with a predetermined Euclidean distance threshold. When a Euclidean distance is less than the threshold, sparsification is performed on the two coordinate data corresponding to that Euclidean distance, deleting one coordinate data and retaining the other.

[0157] Method 2: Based on the distance between each coordinate data in the coordinate dataset corresponding to the current auxiliary line and the current auxiliary line, optimize each coordinate data to obtain an optimized coordinate dataset corresponding to the current auxiliary line.

[0158] In the specific implementation process of this step, for the coordinate dataset corresponding to the current auxiliary line, the vertical distance between each coordinate data in the coordinate dataset and the current auxiliary line can be calculated to obtain the distance value corresponding to each coordinate data. Then, based on each distance value, each coordinate data is filtered to delete coordinate data with a distance value greater than a predetermined distance threshold, thereby obtaining the optimized coordinate dataset after optimization.

[0159] Method 3: Based on the method of exchanging horizontal and vertical coordinates, perform coordinate transformation on each coordinate data in the coordinate dataset corresponding to the current auxiliary line to obtain an optimized coordinate dataset corresponding to the current auxiliary line.

[0160] In this step, for the coordinate dataset corresponding to the current auxiliary line, the horizontal and vertical coordinates of each coordinate data in the dataset can be interchanged. That is, for any coordinate data (a, b), after performing coordinate transformation, transformed coordinate data (b, a) is obtained. This yields the transformed coordinate data corresponding to each coordinate data, allowing for the creation of an optimized coordinate dataset. Subsequently, based on the coordinate data in the optimized dataset, curve fitting can be performed again to obtain the current auxiliary line. Then, based on the refitted current auxiliary line, the current error value is calculated to determine if it is less than or equal to a predetermined error threshold. If it is less than or equal to the predetermined error threshold, the auxiliary line is rotated 90 degrees to obtain the target reversing auxiliary line.

[0161] Based on the above embodiments, taking a coordinate dataset corresponding to any type of auxiliary line as an example, the process of obtaining the target auxiliary line through optimization processing will be explained, such as... Figure 6 As shown, the specific optimization process is as follows:

[0162] Step 1: Collect and obtain the coordinate dataset.

[0163] Step 2: Determine if the data is sufficient. If it is sufficient, proceed to Step 3; otherwise, determine that the auxiliary line optimization has failed and proceed to Step 9 to end the optimization.

[0164] Step 3: Perform initial fitting of auxiliary lines based on the coordinate data in the coordinate dataset to obtain the current auxiliary lines.

[0165] Step 4: Calculate the error based on the current auxiliary line and each coordinate data to obtain the error value corresponding to the current auxiliary line.

[0166] Step 5: Determine if the error value is less than the predetermined error threshold; if not, proceed to step 6; if less, proceed to step 8.

[0167] Step six: Determine if the optimization limit has been reached; if the optimization limit has been reached, proceed to step nine to end the optimization; if the optimization limit has not been reached, proceed to step seven.

[0168] In this step, the optimization limit can be determined based on the number of optimization attempts. For example, if the number of optimization attempts exceeds the predetermined number of attempts, then the optimization limit has been reached. If the error value is not less than the predetermined error threshold and the optimization limit has been reached, it means that the current auxiliary line obtained by optimization does not meet the requirements and cannot be further optimized. Therefore, the optimization fails, and step nine can be executed to end the optimization.

[0169] Step 7: Determine the corresponding optimization method based on the number of optimizations, optimize each coordinate data in the coordinate dataset to obtain the optimized coordinate dataset, obtain the current auxiliary line after optimization, and return to Step 4.

[0170] In this step, the coordinate data in the coordinate dataset corresponding to the current auxiliary line are optimized according to the optimization method corresponding to the number of optimizations, resulting in an optimized coordinate dataset. Subsequently, the current auxiliary line can be refitted based on the coordinate data in the optimized coordinate dataset. The optimization methods corresponding to the number of optimizations are as follows:

[0171] The first optimization method, corresponding to the first optimization, involves optimizing each coordinate data point based on the Euclidean distance between them in the coordinate dataset corresponding to the current auxiliary line, to obtain an optimized coordinate dataset corresponding to the current auxiliary line. Specifically, for the coordinate dataset corresponding to the current auxiliary line, the Euclidean distance (L2 distance) between each coordinate data point in the dataset can be calculated. Based on this Euclidean distance, the coordinate data is sparsified, retaining only one of two coordinate data points with similar Euclidean distances. This yields the optimized coordinate dataset. The current auxiliary line can then be refitted based on the coordinate data in the optimized dataset. Finally, the process returns to step four to determine if the current auxiliary line meets the optimization stopping condition.

[0172] The second optimization method, corresponding to the second optimization, involves optimizing each coordinate data point in the optimized coordinate dataset corresponding to the current auxiliary line based on the distance between that coordinate and the current auxiliary line. This process yields an optimized coordinate dataset for the current auxiliary line. Specifically, the error value of each coordinate data point is determined based on its distance from the current auxiliary line. Then, the coordinate data with the largest error value (or coordinate data with an error value greater than a predetermined distance error value) is deleted. This allows the remaining coordinate data to be used to construct the optimized coordinate dataset. Subsequently, the current auxiliary line can be refitted based on the coordinate data in the optimized coordinate dataset.

[0173] The third optimization method, corresponding to the third optimization, involves transforming the coordinates of each point in the original coordinate dataset to obtain an optimized coordinate dataset corresponding to the current auxiliary line. Subsequently, the current auxiliary line can be refitted based on the transformed coordinates, and the process returns to step four to determine if the current error value of the auxiliary line is less than or equal to a predetermined error threshold. If it is less than or equal to the predetermined error threshold, the auxiliary line can be rotated 90 degrees to obtain the target reversing auxiliary line. In other words, this optimization method adds the coordinate data of previously deleted points to the coordinate dataset to obtain the original coordinate dataset, and then transforms the coordinates of each point in this dataset.

[0174] The fourth optimization method, corresponding to the fourth optimization, involves optimizing each transformed coordinate data point (transformed coordinate data) in the optimized coordinate dataset after the third optimization to obtain an optimized coordinate dataset corresponding to the current auxiliary line. In other words, for the optimized coordinate dataset after the third optimization, the Euclidean distance (L2 distance) between each transformed coordinate data point in the dataset can be calculated, and the transformed coordinate data can be sparsified based on this Euclidean distance to obtain the optimized coordinate dataset after the fourth optimization. The principle of this optimization method is similar to that of the first optimization method and will not be elaborated further here.

[0175] The fifth optimization method, corresponding to the fifth optimization, involves optimizing each transformed coordinate data point (transformed coordinate data) in the optimized coordinate dataset after the fourth optimization, based on the distance between these data points and the current auxiliary line. This yields an optimized coordinate dataset corresponding to the current auxiliary line. In other words, for the optimized coordinate dataset after the fourth optimization, the vertical distance between each transformed coordinate data point and the current auxiliary line can be calculated. Based on these distance values, the coordinate data is then filtered to obtain the optimized coordinate dataset. The principle behind this optimization method is similar to that of the second optimization method and will not be elaborated further here.

[0176] Step 8: Set the current guide line as the target reversing guide line and save it.

[0177] Step 9: End the current auxiliary line optimization.

[0178] In this embodiment, the above optimization steps can be used to optimize the auxiliary lines for coordinate datasets of each location type to generate corresponding target reversing auxiliary lines, thereby obtaining target reversing auxiliary lines corresponding to each location type and solving the problem of inaccurate straight-line reversing auxiliary lines caused by distortion of wide-angle and other camera imaging.

[0179] Another embodiment of this application provides an obstacle warning device for reversing assistance, such as... Figure 7 As shown, it includes:

[0180] Module 11 is used to construct several target reversing auxiliary lines based on historical reversing images containing target reference objects, and to determine the target position of each target reversing auxiliary line in the historical reversing image;

[0181] The recognition module 12 is used to identify obstacles in the real-time captured current reversing image and determine several obstacle objects;

[0182] The first acquisition module 13 is used to acquire the current position of each obstacle object in the current reversing image;

[0183] The early warning module 14 is used to perform early warning detection on each of the obstacle objects based on the target position of each of the target reversing auxiliary lines and the current position of each of the objects, so as to provide obstacle warning based on the detection results.

[0184] In this embodiment, during the specific real-time process, the construction module includes an acquisition unit and a construction unit. The acquisition unit is used to: respond to the user's selection operation of the target reference object in the historical reversing image and the selection operation of each pixel region in the historical reversing image, and obtain coordinate data corresponding to each pixel region; the construction unit is used to: construct and obtain a number of target reversing auxiliary lines based on the coordinate data.

[0185] In this embodiment, during real-time operation, the obstacle warning device for reversing assistance further includes a configuration module. The configuration module is used to: respond to the selection command for the auxiliary line type for each coordinate data, and configure the corresponding auxiliary line type for each coordinate data to obtain a coordinate dataset corresponding to each auxiliary line type. The construction unit includes a first construction unit and a second construction unit. The first construction unit is used to: generate a target reversing auxiliary line of straight line type corresponding to the auxiliary line type based on each coordinate data in the coordinate dataset corresponding to the same auxiliary line type, to obtain several target reversing auxiliary lines; or, the second construction unit is used to: generate a target reversing auxiliary line of curve type corresponding to the auxiliary line type based on each coordinate data in the coordinate dataset corresponding to the same auxiliary line type using an optimization processing method, to obtain several target reversing auxiliary lines.

[0186] In this embodiment, the auxiliary line type includes any one or more of the following types: left auxiliary line type along the vehicle length direction, right auxiliary line type along the vehicle length direction, first auxiliary line type along the vehicle width direction, second auxiliary line type along the vehicle width direction, and third auxiliary line type along the vehicle width direction; the distance between the target reversing auxiliary line corresponding to the first auxiliary line type, the second auxiliary line type, and the third auxiliary line type and the rear of the vehicle is different.

[0187] In this embodiment, the second construction unit specifically includes a fitting subunit, a calculation subunit, and an optimization subunit. The fitting subunit is used to: perform curve fitting based on the coordinate data in the coordinate dataset corresponding to the same auxiliary line type to obtain the current auxiliary line corresponding to the auxiliary line type. The calculation subunit is used to: calculate the current error value corresponding to each current auxiliary line based at least on each current auxiliary line. The optimization subunit is used to: optimize the corresponding current auxiliary line based on the current error value of the current auxiliary line until a predetermined optimization stopping condition is met, and use the current auxiliary line as the target reversing auxiliary line to obtain the target reversing auxiliary line corresponding to each auxiliary line type.

[0188] In this embodiment, the calculation subunit is specifically used to: calculate the distance between each coordinate data and the current auxiliary line based on each current auxiliary line and each coordinate data in the coordinate dataset corresponding to each current auxiliary line; and calculate the current error value of the current auxiliary line based on each distance value.

[0189] In this embodiment, the optimization subunit is specifically used to: determine whether the current auxiliary line meets the optimization stopping condition based at least on the current error value of the current auxiliary line and a predetermined error threshold; if the current auxiliary line does not meet the optimization stopping condition, optimize each coordinate data in the coordinate dataset corresponding to the current auxiliary line according to a predetermined optimization method to obtain an optimized coordinate dataset, and refit the current auxiliary line based on each coordinate data in the optimized coordinate dataset; if the current auxiliary line meets the optimization stopping condition, use the current auxiliary line as the target reversing auxiliary line.

[0190] In this embodiment, the optimization subunit is specifically used to: compare the current error value with a predetermined error threshold; determine that the optimization stop condition is met when the current error value is less than or equal to the error threshold or the number of optimizations reaches a predetermined number of times; and determine that the optimization stop condition is not met when the current error value is greater than the error threshold and the number of optimizations has not reached the predetermined number of times.

[0191] In the specific implementation of this embodiment, the predetermined optimization method includes any one or more of the following methods:

[0192] Based on the coordinate data of the current auxiliary line and the Euclidean distance between each coordinate data, each coordinate data is optimized to obtain an optimized coordinate data set corresponding to the current auxiliary line.

[0193] And / or, based on the coordinate data of the current auxiliary line and the distance between each coordinate data and the current auxiliary line, optimize each coordinate data to obtain an optimized coordinate data set corresponding to the current auxiliary line;

[0194] And / or, based on the method of exchanging horizontal and vertical coordinates, perform coordinate transformation on each coordinate data in the coordinate dataset corresponding to the current auxiliary line to obtain an optimized coordinate dataset corresponding to the current auxiliary line.

[0195] In this embodiment, the target reference object includes a target calibration field, and the obstacle warning device for reversing assistance also includes a drawing module. The drawing module is used to: draw n*n squares in the predetermined parking area according to a preset side length to pre-arrange and obtain the target calibration field.

[0196] In this embodiment, the obstacle warning device for reversing assistance further includes a second acquisition module. The second acquisition module is used to: determine the object category of each obstacle object before performing warning detection on each obstacle object based on the target position of each target reversing auxiliary line and each current position, and obtain the target warning level corresponding to each obstacle based on the object category of each obstacle object.

[0197] The warning module is specifically used to: obtain the warning range corresponding to the warning level of each target based on the target position of each target reversing auxiliary line, so as to obtain the warning range corresponding to each obstacle object; perform warning detection on each obstacle object based on the warning range corresponding to each obstacle object and the current position of each obstacle object, so as to determine whether each obstacle object meets the corresponding warning conditions, and perform obstacle warning when the warning conditions are met.

[0198] In this embodiment, the obstacle warning device for reversing assistance further includes a third acquisition module, a position determination module, a parameter determination module, and a prompting module; the third acquisition module is used to: acquire adjacent reversing images adjacent to the shooting time based on the shooting time of the current reversing image;

[0199] The location determination module is used to: determine the historical location of each obstacle object in the adjacent reversing images;

[0200] The parameter determination module is used to: determine the motion parameters of each obstacle object based on its historical and current positions;

[0201] The prompting module is used to: provide warning information prompts for each obstacle object in the current reversing image based on the motion parameters of each obstacle object.

[0202] In this embodiment, the obstacle warning device for reversing assistance can easily and quickly construct reversing guide lines by utilizing historical reversing images containing target reference objects. Subsequently, based on the target position of the reversing guide lines in the reversing image, the device can detect the position of each obstacle in the reversing image and perform obstacle warning. This realizes obstacle warning by using reversing images, that is, only a single monocular camera is needed to capture reversing images to achieve obstacle warning detection. There is no need to detect the actual distance between the obstacle and the vehicle, so no other detection devices are required, reducing the use of detection devices and lowering the cost of obstacle warning.

[0203] Another embodiment of this application provides a storage medium storing a computer program, which, when executed by a processor, implements the following method steps:

[0204] Step 1: Based on historical reversing images containing target reference objects, construct several target reversing auxiliary lines and determine the target position of each target reversing auxiliary line in the historical reversing images;

[0205] Step 2: Perform obstacle recognition on the real-time captured image of the reversing vehicle to identify several obstacle objects;

[0206] Step 3: Obtain the current position of each obstacle object in the current reversing image;

[0207] Step 4: Based on the target position of each of the target reversing auxiliary lines and the current position of each of the targets, perform early warning detection on each of the obstacle objects, and provide obstacle warning based on the detection results.

[0208] The specific implementation process of the above method steps can be found in any of the above embodiments of the obstacle warning method for reversing assistance, and will not be repeated here.

[0209] The storage medium in this application can conveniently and quickly construct reversing guide lines by utilizing historical reversing images containing target reference objects. Subsequently, based on the target position of the reversing guide lines in the reversing image, the position of each obstacle in the reversing image can be detected and obstacle warnings can be performed. This realizes obstacle warning by using reversing images. That is, only a single monocular camera is needed to capture reversing images to achieve obstacle warning detection. There is no need to detect the actual distance between the obstacle and the vehicle. Therefore, no other detection devices are needed, reducing the use of detection devices and lowering the cost of obstacle warning.

[0210] Another embodiment of this application provides an electronic device, such as... Figure 8 As shown, it includes at least a memory and a processor. The memory stores a computer program, and the processor, when executing the computer program in the memory, implements the following method steps:

[0211] Step 1: Based on historical reversing images containing target reference objects, construct several target reversing auxiliary lines and determine the target position of each target reversing auxiliary line in the historical reversing images;

[0212] Step 2: Perform obstacle recognition on the real-time captured image of the reversing vehicle to identify several obstacle objects;

[0213] Step 3: Obtain the current position of each obstacle object in the current reversing image;

[0214] Step 4: Based on the target position of each of the target reversing auxiliary lines and the current position of each of the targets, perform early warning detection on each of the obstacle objects, and provide obstacle warning based on the detection results.

[0215] The specific implementation process of the above method steps can be found in any of the above embodiments of the obstacle warning method for reversing assistance, and will not be repeated here.

[0216] The electronic device in this application can easily and quickly construct reversing guide lines by utilizing historical reversing images containing target reference objects. Subsequently, based on the target position of the reversing guide lines in the reversing image, the device can detect the position of each obstacle in the reversing image and perform obstacle warning. This realizes obstacle warning by using reversing images. That is, only a single monocular camera is needed to capture reversing images to achieve obstacle warning detection. There is no need to detect the actual distance between the obstacle and the vehicle. Therefore, no other detection devices are needed, reducing the use of detection devices and lowering the cost of obstacle warning.

[0217] The above embodiments are merely exemplary embodiments of this application and are not intended to limit this application. The scope of protection of this application is defined by the claims. Those skilled in the art can make various modifications or equivalent substitutions to this application within its substance and scope of protection, and such modifications or equivalent substitutions should also be considered to fall within the scope of protection of this application.

Claims

1. An obstacle warning method for reversing assistance, characterized in that, include: Based on historical reversing images containing target reference objects, several target reversing auxiliary lines are constructed, and the target position of each target reversing auxiliary line in the historical reversing image is determined. Obstacle recognition is performed on the real-time captured image of the vehicle reversing to identify several obstacle objects; Obtain the current position of each obstacle object in the current reversing image; Based on the target position of each of the target reversing auxiliary lines and the current position of each of the objects, a warning detection is performed on each of the obstacles, and an obstacle warning is given based on the detection results. The target reference object includes: a target calibration field plotted with n×n squares; The process of constructing several target reversing auxiliary lines based on historical reversing images containing target reference objects specifically includes: In response to the user's selection operation of the target reference object in the historical reversing image and the selection operation of each pixel region in the historical reversing image, the coordinate data corresponding to each pixel region is obtained; Based on the coordinate data in the coordinate dataset corresponding to the same auxiliary line type, curve fitting is performed to obtain the current auxiliary line corresponding to the auxiliary line type. Based on each current auxiliary line and each coordinate data in the coordinate dataset corresponding to each current auxiliary line, calculate the distance value between each coordinate data and the current auxiliary line; Based on the distance values ​​corresponding to the current auxiliary line, the current error value of the current auxiliary line is calculated using the average, variance, or standard deviation. Based on the current error value of the current auxiliary line, the corresponding current auxiliary line is optimized until the predetermined optimization stopping condition is met. The current auxiliary line is then used as the target reversing auxiliary line to obtain the target reversing auxiliary line corresponding to each auxiliary line type.

2. The method as described in claim 1, characterized in that, Before constructing and obtaining several target reversing auxiliary lines based on the coordinate data, the method further includes: For each coordinate data, respond to the auxiliary line type selection command to configure the corresponding auxiliary line type for each coordinate data to obtain the coordinate dataset corresponding to each auxiliary line type.

3. The method as described in claim 2, characterized in that, The auxiliary line types include any one or more of the following types: left auxiliary line type along the vehicle length direction, right auxiliary line type along the vehicle length direction, first auxiliary line type along the vehicle width direction, second auxiliary line type along the vehicle width direction, and third auxiliary line type along the vehicle width direction; the distance between the target reversing auxiliary line corresponding to the first auxiliary line type, the second auxiliary line type, and the third auxiliary line type and the rear of the vehicle is different.

4. The method as described in claim 1, characterized in that, The optimization process based on the current error value of the current auxiliary line is performed on the corresponding current auxiliary line until a predetermined optimization stopping condition is met. The current auxiliary line is then used as the target reversing auxiliary line to obtain target reversing auxiliary lines corresponding to each auxiliary line type, including: Based at least on the current error value of the current auxiliary line and the predetermined error threshold, determine whether the current auxiliary line meets the optimization stopping condition; If it is determined that the current auxiliary line does not meet the optimization stopping condition, the coordinate data in the coordinate dataset corresponding to the current auxiliary line are optimized according to the predetermined optimization method to obtain an optimized coordinate dataset, and the current auxiliary line is refitted based on the coordinate data in the optimized coordinate dataset. If the current auxiliary line is determined to meet the optimized stopping conditions, the current auxiliary line will be used as the target reversing auxiliary line.

5. The method as described in claim 4, characterized in that, The step of determining whether the current auxiliary line meets the optimization stopping condition, based at least on the current error value of the current auxiliary line and a predetermined error threshold, includes: Compare the current error value with a predetermined error threshold; If the current error value is less than or equal to the error threshold, or the number of optimization attempts reaches a predetermined number, it is determined that the optimization stopping condition is met. If the current error value is greater than the error threshold and the number of optimization attempts has not reached the predetermined number, it is determined that the optimization stop condition is not met.

6. The method as described in claim 4, characterized in that, The predetermined optimization method includes any one or more of the following methods: Based on the coordinate data of the current auxiliary line and the Euclidean distance between each coordinate data, each coordinate data is optimized to obtain an optimized coordinate data set corresponding to the current auxiliary line. And / or, based on the coordinate data of the current auxiliary line and the distance between each coordinate data and the current auxiliary line, optimize each coordinate data to obtain an optimized coordinate data set corresponding to the current auxiliary line; And / or, based on the method of exchanging horizontal and vertical coordinates, perform coordinate transformation on each coordinate data in the coordinate dataset corresponding to the current auxiliary line to obtain an optimized coordinate dataset corresponding to the current auxiliary line.

7. The method as described in claim 1, characterized in that, The target reference object includes a target calibration field, and the method further includes: Draw n within the designated parking area according to the preset side length. n squares are pre-arranged to obtain the target calibration field.

8. The method as described in claim 1, characterized in that, Before performing warning detection on each obstacle object based on the target position of each target reversing auxiliary line and each current position, the method further includes: Determine the object category of each obstacle object, and obtain the target warning level corresponding to each obstacle based on the object category of each obstacle object; The method of detecting and warning of obstacles based on the target positions of the reversing auxiliary lines and the current positions of the obstacles includes: Based on the target position of each target reversing auxiliary line, obtain the warning range corresponding to each target warning level, so as to obtain the warning range corresponding to each obstacle object; Based on the warning range corresponding to each obstacle object and the current position of each obstacle object, a warning detection is performed on each obstacle object to determine whether each obstacle object meets the corresponding warning conditions, and an obstacle warning is issued when the warning conditions are met.

9. The method as described in claim 1, characterized in that, The method further includes: Based on the shooting time of the current reversing image, obtain the adjacent reversing images that are adjacent to the shooting time; Determine the historical position of each obstacle object in the adjacent reversing images; Based on the historical and current positions of each obstacle object, determine the motion parameters of each obstacle object; Based on the motion parameters of each obstacle, warning information is provided for each obstacle in the current reversing image.

10. An obstacle warning device for reversing assistance, characterized in that, include: The module is used to respond to the user's selection operation of each pixel region in the historical reversing image based on the target reference object in the historical reversing image, obtain the coordinate data corresponding to each pixel region; based on each coordinate data and the auxiliary line type corresponding to each coordinate data, construct several target reversing auxiliary lines of straight line type or curve type, and determine the target position of each target reversing auxiliary line in the historical reversing image. The target reference object includes: a target calibration field plotted with n×n squares; The recognition module is used to identify obstacles in the real-time captured image of the current reversing vehicle and determine several obstacle objects. The first acquisition module is used to acquire the current position of each obstacle object in the current reversing image; The early warning module is used to perform early warning detection on each of the obstacle objects based on the target position of each of the target reversing auxiliary lines and the current position of each of the objects, so as to provide obstacle warning based on the detection results; The building module is specifically used for: Based on the coordinate data in the coordinate dataset corresponding to the same auxiliary line type, curve fitting is performed to obtain the current auxiliary line corresponding to the auxiliary line type. Based on each current auxiliary line and each coordinate data in the coordinate dataset corresponding to each current auxiliary line, calculate the distance value between each coordinate data and the current auxiliary line; Based on each of the aforementioned distance values, the current error value of the current auxiliary line is calculated using the average, variance, or standard deviation method. Based on the current error value of the current auxiliary line, the corresponding current auxiliary line is optimized until the predetermined optimization stopping condition is met. The current auxiliary line is then used as the target reversing auxiliary line to obtain the target reversing auxiliary line corresponding to each auxiliary line type.

11. A storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the steps of the obstacle warning method for reversing assistance as described in any one of claims 1-7.

12. An electronic device, characterized in that, It includes at least a memory and a processor, wherein the memory stores a computer program, and the processor, when executing the computer program in the memory, implements the steps of the obstacle warning method for reversing assistance as described in any one of claims 1-7.

Images