Obstacle Recognition Method and Device, and Driving Method and Device

Abstract

This specification provides an obstacle recognition method and apparatus, as well as a driving method and apparatus. The obstacle recognition method includes: obtaining a to-be-processed image of a to-be-processed area captured by a camera; performing image recognition on the to-be-processed image using a target recognition model to obtain a first recognition result; wherein the first recognition result includes position information of a first obstacle; obtaining current position information of a second obstacle in the to-be-processed area; wherein the second obstacle is an obstacle determined within a preset time period before the current moment; and determining a target obstacle based on the position information of the first obstacle and the current position information of the second obstacle, thereby improving the accuracy of obstacle recognition.

Description

Technical Field

[0001] This specification relates to the field of autonomous driving technology, and particularly to methods and devices for obstacle recognition and driving methods and devices. Background Art

[0002] With the progress of technology, driverless technology has received increasing attention. Among them, low obstacle recognition is an important part of autonomous driving technology.

[0003] Currently, when recognizing low obstacles, generally, the images captured by the on-vehicle camera of the vehicle are used for recognition, that is, relevant models or algorithms are used to recognize the images captured by the on-vehicle camera to determine the low obstacles in the images and the position information of the low obstacles, so as to obtain the situation of low obstacles around the current position of the vehicle.

[0004] However, since only a single-frame image is used to determine low obstacles during low obstacle recognition, it is easy to miss low obstacles, resulting in a low accuracy rate of obstacle recognition. Summary of the Invention

[0005] To overcome the problems existing in the related art, this specification provides methods and devices for obstacle recognition and driving methods and devices.

[0006] According to the first aspect of the embodiments of this specification, an obstacle recognition method is provided, and the method includes:

[0007] Obtain a to-be-processed image of a to-be-processed area captured by a camera;

[0008] Perform image recognition on the to-be-processed image using a target recognition model to obtain a first recognition result; wherein, the first recognition result includes the position information of a first obstacle;

[0009] Obtain the current position information of a second obstacle in the to-be-processed area; wherein, the second obstacle is an obstacle determined within a preset time period before the current moment;

[0010] Determine a target obstacle according to the position information of the first obstacle and the current position information of the second obstacle.

[0011] In a possible design, the target obstacle includes a low obstacle;

[0012] The determining the target obstacle according to the position information of the first obstacle and the current position information of the second obstacle includes:

[0013] Determine an association matrix between the second obstacle and the first obstacle according to the current position information of the second obstacle and the position information of the first obstacle;

[0014] Obtain the target association result between the second obstacle and the first obstacle according to the association matrix;

[0015] Update the second obstacle according to the target association result, and determine the low obstacle according to the updated second obstacle.

[0016] In a possible design, the updating the second obstacle according to the target association result includes:

[0017] When it is determined that the target association result is that the second obstacle is successfully associated with the first obstacle, update the current position information of the second obstacle corresponding to the target association result according to the position information of the first obstacle corresponding to the target association result;

[0018] When it is determined that the target association result is that the association of the first obstacle fails, use the first obstacle corresponding to the target association result as the new second obstacle;

[0019] When it is determined that the target association result is that the association of the second obstacle fails, update the second obstacle corresponding to the target association result.

[0020] In a possible design, the updating the second obstacle corresponding to the target association result includes:

[0021] Obtain the status information of the second obstacle corresponding to the target association result;

[0022] Judge whether the status information meets the preset deletion condition;

[0023] When it is determined that the status information meets the preset deletion condition, delete the second obstacle corresponding to the target association result;

[0024] When it is determined that the status information does not meet the preset deletion condition, update the status information.

[0025] In a possible design, the status information includes the recognition failure ratio, and the judging whether the status information meets the preset deletion condition includes:

[0026] Judge whether the recognition failure ratio is greater than the preset ratio threshold;

[0027] If it is greater than the preset ratio threshold, determine that the status information meets the preset deletion condition;

[0028] If it is less than or equal to the preset ratio threshold, determine that the status information does not meet the preset deletion condition.

[0029] In a possible design, the current position information of the second obstacle includes the current position information of the second obstacle corresponding to at least one preset tracking algorithm;

[0030] Determining the association matrix between the second obstacle and the first obstacle according to the current position information of the second obstacle and the position information of the first obstacle includes:

[0031] Obtain the priority corresponding to each preset tracking algorithm;

[0032] Select a target tracking algorithm from all preset tracking algorithms according to the sorting from high to low priority, and obtain the current position information of the second obstacle corresponding to the target tracking algorithm;

[0033] Determine the association matrix corresponding to the target tracking algorithm according to the current position information of the second obstacle corresponding to the target tracking algorithm and the position information of the first obstacle.

[0034] In a possible design, obtaining the target association result between the second obstacle and the first obstacle according to the association matrix includes:

[0035] Determine the initial association result between the second obstacle corresponding to the target tracking algorithm and the first obstacle according to the association matrix corresponding to the target tracking algorithm;

[0036] If there are failures in associating the first obstacle and failures in associating the second obstacle in the initial association result, obtain the number of determined association matrices, and determine whether the number of determined association matrices is equal to the number of preset tracking algorithms;

[0037] If it is equal to the number of preset tracking algorithms, perform a merging process on all initial association results to obtain the target association result;

[0038] If it is not equal to the number of preset tracking algorithms, continue to select a target tracking algorithm from all preset tracking algorithms according to the sorting from high to low priority, and determine the association matrix corresponding to the target tracking algorithm according to the current position information of the remaining second obstacle and the position information of the remaining first obstacle corresponding to the target tracking algorithm; wherein, the initial association result corresponding to the remaining second obstacle is a failure in associating the second obstacle, and the initial association result corresponding to the remaining first obstacle is a failure in associating the first obstacle.

[0039] In a possible design, before obtaining the current position information of the second obstacle, it further includes:

[0040] Obtain the historical position information of the second obstacle;

[0041] For each preset tracking algorithm, based on the historical position information of the second obstacle and the preset tracking algorithm, perform position prediction on the second obstacle to obtain the current position information of the second obstacle corresponding to the preset tracking algorithm.

[0042] In a possible design, the method further includes:

[0043] Update the electronic map corresponding to the area to be processed according to the target obstacle; wherein, the target obstacle is an obstacle with a height less than a preset threshold.

[0044] In a possible design, the position information of the first obstacle includes size information determined according to the set of edge pixel points of the first obstacle and / or the position information of the rectangular area corresponding to the first obstacle.

[0045] According to the second aspect of the embodiments of this specification, a driving method is provided, which is applied to a vehicle. The method includes:

[0046] Obtain an updated electronic map; wherein, the updated electronic map is obtained according to the method described in the first aspect and various possible designs of the first aspect;

[0047] Determine a driving route based on the updated electronic map, so that the vehicle performs autonomous driving according to the driving route.

[0048] According to the third aspect of the embodiments of this specification, an obstacle recognition device is provided, including:

[0049] An image acquisition module, configured to acquire a to-be-processed image of the area to be processed captured by a camera;

[0050] An image processing module, configured to perform image recognition on the to-be-processed image by using a target recognition model to obtain a first recognition result; wherein, the first recognition result includes the position information of the first obstacle;

[0051] An information acquisition module, configured to acquire the current position information of a second obstacle in the area to be processed; wherein, the second obstacle is an obstacle determined within a preset time period before the current moment;

[0052] An obstacle recognition module, configured to determine a target obstacle according to the position information of the first obstacle and the current position information of the second obstacle.

[0053] In a possible design, the target obstacle includes a low obstacle;

[0054] The obstacle recognition module includes:

[0055] An association matrix determination unit, configured to determine an association matrix between the second obstacle and the first obstacle according to the current position information of the second obstacle and the position information of the first obstacle;

[0056] An association result determination unit, configured to obtain a target association result between the second obstacle and the first obstacle according to the association matrix;

[0057] An obstacle determination unit, configured to perform an update process on the second obstacle according to the target association result, and determine the low obstacle according to the updated second obstacle.

[0058] In a possible design, the obstacle determination unit is specifically configured to:

[0059] When determining that the target association result is that the second obstacle is successfully associated with the first obstacle, update the current position information of the second obstacle corresponding to the target association result according to the position information of the first obstacle corresponding to the target association result;

[0060] When determining that the target association result is that the first obstacle association fails, use the first obstacle corresponding to the target association result as the new second obstacle;

[0061] When determining that the target association result is that the second obstacle association fails, perform an update process on the second obstacle corresponding to the target association result.

[0062] In a possible design, the obstacle determination unit is specifically configured to:

[0063] Obtain the status information of the second obstacle corresponding to the target association result;

[0064] Judge whether the status information meets a preset deletion condition;

[0065] When determining that the status information meets the preset deletion condition, delete the second obstacle corresponding to the target association result;

[0066] When determining that the status information does not meet the preset deletion condition, update the status information.

[0067] In a possible design, the obstacle determination unit is specifically configured to:

[0068] Judge whether the recognition failure ratio is greater than a preset ratio threshold;

[0069] If it is greater than the preset ratio threshold, determine that the status information meets the preset deletion condition;

[0070] If it is less than or equal to the preset ratio threshold, it is determined that the status information does not meet the preset deletion condition.

[0071] In a possible design, the current position information of the second obstacle includes the current position information of the second obstacle corresponding to at least one preset tracking algorithm;

[0072] The correlation matrix determination unit is specifically configured to:

[0073] Obtain the priority corresponding to each preset tracking algorithm;

[0074] Select a target tracking algorithm from all the preset tracking algorithms according to the sorting from high to low priority, and obtain the current position information of the second obstacle corresponding to the target tracking algorithm;

[0075] Determine the correlation matrix corresponding to the target tracking algorithm according to the current position information of the second obstacle corresponding to the target tracking algorithm and the position information of the first obstacle.

[0076] In a possible design, the correlation result determination unit is specifically configured to:

[0077] Determine the initial correlation result between the second obstacle corresponding to the target tracking algorithm and the first obstacle according to the correlation matrix corresponding to the target tracking algorithm;

[0078] If there are cases where the first obstacle fails to be correlated and the second obstacle fails to be correlated in the initial correlation result, obtain the number of determined correlation matrices, and determine whether the number of determined correlation matrices is equal to the number of preset tracking algorithms;

[0079] If it is equal to the number of preset tracking algorithms, perform a merging process on all the initial correlation results to obtain a target correlation result;

[0080] If it is not equal to the number of preset tracking algorithms, continue to select a target tracking algorithm from all the preset tracking algorithms according to the sorting from high to low priority, and determine the correlation matrix corresponding to the target tracking algorithm according to the current position information of the remaining second obstacle and the position information of the remaining first obstacle corresponding to the target tracking algorithm; wherein, the initial correlation result corresponding to the remaining second obstacle is that the second obstacle fails to be correlated, and the initial correlation result corresponding to the remaining first obstacle is that the first obstacle fails to be correlated.

[0081] In a possible design, the information acquisition module is further configured to:

[0082] Before obtaining the current position information of the second obstacle, obtain the historical position information of the second obstacle;

[0083] For each preset tracking algorithm, based on the historical position information of the second obstacle and the preset tracking algorithm, perform position prediction on the second obstacle to obtain the current position information of the second obstacle corresponding to the preset tracking algorithm.

[0084] In a possible design, the obstacle recognition device further includes:

[0085] A map update module, configured to update the electronic map corresponding to the area to be processed according to the target obstacle; wherein, the target obstacle is an obstacle with a height less than a preset threshold.

[0086] In a possible design, the position information of the first obstacle includes size information determined according to the set of edge pixel points of the first obstacle and / or the position information of the rectangular area corresponding to the first obstacle.

[0087] According to the fourth aspect of the embodiments of the present specification, a driving device is provided, which is applied to a vehicle, and the device includes:

[0088] A map acquisition module, configured to acquire the updated electronic map; wherein, the updated electronic map is obtained by the method according to the first aspect and various possible designs of the first aspect;

[0089] An automatic driving module, configured to determine a driving route based on the updated electronic map, so that the vehicle performs automatic driving according to the driving route.

[0090] According to the fifth aspect of the embodiments of the present specification, a computer device is provided, including:

[0091] A processor;

[0092] A memory for storing instructions executable by the processor;

[0093] Wherein, the processor is configured to:

[0094] Acquire a to-be-processed image of the area to be processed captured by a camera;

[0095] Perform image recognition on the to-be-processed image using a target recognition model to obtain a first recognition result; wherein, the first recognition result includes the position information of the first obstacle;

[0096] Acquire the current position information of the second obstacle in the area to be processed; wherein, the second obstacle is an obstacle determined within a preset time period before the current moment;

[0097] Determine a target obstacle according to the position information of the first obstacle and the current position information of the second obstacle.

[0098] According to a sixth aspect of the embodiments of the present specification, a vehicle-mounted terminal is provided, including:

[0099] A processor;

[0100] A memory for storing instructions executable by the processor;

[0101] Wherein, the processor is configured to:

[0102] Obtain an updated electronic map; wherein, the updated electronic map is obtained by the method according to the first aspect and various possible designs of the first aspect;

[0103] Determine a driving route based on the updated electronic map, and drive according to the driving route.

[0104] According to a seventh aspect of the embodiments of the present specification, a computer-readable storage medium is provided. Computer-executable instructions are stored in the computer-readable storage medium. When the processor executes the computer-executable instructions, the obstacle recognition method according to the first aspect and various possible designs of the first aspect is implemented.

[0105] According to an eighth aspect of the embodiments of the present specification, a computer-readable storage medium is provided. Computer-executable instructions are stored in the computer-readable storage medium. When the processor executes the computer-executable instructions, the driving method according to the second aspect and various possible designs of the second aspect is implemented.

[0106] According to a ninth aspect of the embodiments of the present specification, a computer program product is provided, including a computer program. When the computer program is executed by the processor, the obstacle recognition method according to the first aspect and various possible designs of the first aspect is implemented.

[0107] According to a tenth aspect of the embodiments of the present specification, a computer program product is provided, including a computer program. When the computer program is executed by the processor, the driving method according to the second aspect and various possible designs of the second aspect is implemented.

[0108] The technical solutions provided by the embodiments of the present specification may include the following beneficial effects:

[0109] In the embodiments of this specification, when a to-be-processed image of a to-be-processed area captured by a camera is obtained, it indicates that obstacles in the to-be-processed area need to be recognized. Then, a target recognition model is used to perform image recognition on the to-be-processed image to obtain a first recognition result, that is, to determine the position information of the first obstacle in the to-be-processed image. After determining the first obstacle, the position information of the obstacles determined within a preset time period before the current moment, that is, the second obstacle, and the position information of the first obstacle are comprehensively used to determine the obstacle corresponding to the to-be-processed image, that is, the target obstacle is comprehensively determined according to the second obstacle and the first obstacle, rather than directly taking the first obstacle as the obstacle corresponding to the to-be-processed area, that is, the target obstacle is not determined only based on a single-frame image, which can improve the accuracy of obstacle recognition.

[0110] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit this specification. Brief Description of the Drawings

[0111] The accompanying drawings here are incorporated into the specification and constitute a part of this specification, showing the embodiments consistent with this specification, and are used together with the specification to explain the principles of this specification.

[0112] Figure 1 is a schematic diagram of an obstacle acquisition system shown according to an exemplary embodiment of this specification.

[0113] Figure 2 is a flowchart of an obstacle recognition method shown according to an exemplary embodiment of this specification.

[0114] Figure 3 is a flowchart of another obstacle recognition method shown according to an exemplary embodiment of this specification.

[0115] Figure 4 is a flowchart of a driving method shown according to an exemplary embodiment of this specification.

[0116] Figure 5 is a hardware structure diagram of a computer device where the obstacle recognition device in the embodiments of this specification is located.

[0117] Figure 6 is a hardware structure diagram of an in-vehicle terminal where the driving device in the embodiments of this specification is located.

[0118] Figure 7 is a structural block diagram of an obstacle recognition device shown according to an exemplary embodiment of this specification.

[0119] Figure 8 is a structural block diagram of a driving device shown according to an exemplary embodiment of this specification. Detailed Embodiments

[0120] Exemplary embodiments will be described in detail herein, and examples thereof are shown in the accompanying drawings. When the following description refers to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this specification. On the contrary, they are merely examples of devices and methods consistent with some aspects of this specification as detailed in the appended claims.

[0121] The terms used in this specification are for the purpose of describing particular embodiments only and are not intended to limit this specification. The singular forms "a", "the", and "said" used in this specification and the appended claims are also intended to include the plural forms unless the context clearly dictates otherwise. It should also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0122] It should be understood that although the terms first, second, third, etc. may be used in this specification to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other. For example, without departing from the scope of this specification, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "when" or "while" or "in response to determining".

[0123] Next, the embodiments of this specification will be described in detail.

[0124] In the prior art, in order to improve the accuracy of the electronic map used for automatic vehicle driving to ensure the obstacle avoidance ability of the vehicle, it is necessary to determine, that is, to identify low obstacles in the environment, and the identification process is as Figure 1 shown. During the process of the vehicle 101 driving on the road surface in the acquisition area, the camera 102 placed on the acquisition vehicle 101 takes pictures to obtain an image containing the surrounding environment. The electronic device 103 identifies the low obstacles on the image and their corresponding position and size information, so as to update the electronic map by using the identified low obstacles. However, when identifying low obstacles, determining low obstacles only based on a single-frame image is likely to cause actual low obstacles not to be photographed due to reasons such as shooting angle and weather, resulting in the problem of missing low obstacles and reducing the accuracy of obstacle determination. At the same time, when identifying low obstacles on the image, a two-dimensional rectangular frame is used to describe the low obstacles, that is, the circumscribed rectangle of the low obstacles is drawn, resulting in the size information of the low obstacles determined according to this circumscribed rectangle being too large, affecting the obstacle avoidance ability and route planning of the vehicle.

[0125] Among them, the electronic device can be an on-vehicle terminal on a collection vehicle, or can also be a terminal device such as a computer or a server. Here, it is not limited thereto.

[0126] Therefore, in view of the above problems, the technical concept of this application is that when identifying low obstacles on the current image frame, based on the attribute information of the low obstacles on the currently identified current image frame, that is, the position and size information, and in combination with the position tracking results of the low obstacles determined within a period of time before the current moment, to jointly determine the low obstacles corresponding to the current image frame, rather than directly taking only the low obstacles identified on the current image frame as the low obstacles corresponding to the current image frame, so as to accurately identify low obstacles, avoid the problem of missing low obstacles, and improve the accuracy of obstacle determination. At the same time, the attribute information of the low obstacles, that is, the size information, is determined according to the pixel points on the edge contour of the low obstacles, ensuring the accuracy of the attribute information, thereby effectively ensuring the obstacle avoidance ability of the vehicle and enabling the vehicle to better plan its route.

[0127] Such as Figure 2 shown, Figure 2 is a flowchart of an obstacle recognition method shown in this specification according to an exemplary embodiment. The execution subject of this method is a computer device. For example, Figure 1 the electronic device in

[0128] Step S201, obtain a to-be-processed image of a to-be-processed area captured by a camera.

[0129] In this embodiment, an image of the environment where the collection vehicle is located captured by the camera is obtained, and this image is used as the to-be-processed image for identifying obstacles in the captured area corresponding to this to-be-processed image, that is, in the to-be-processed area.

[0130] Among them, the obstacle can be a low obstacle or other types of obstacles. Here, it is not limited thereto. Among them, a low obstacle refers to an obstacle with a height less than a preset threshold. Among them, the preset threshold can be set according to the actual situation. Here, it is not limited thereto.

[0131] Optionally, the camera, that is, the on-vehicle camera, can be installed on the collection vehicle. During the driving process of the collection vehicle, the camera captures the surrounding environment to obtain corresponding images.

[0132] Step S202, perform image recognition on the to-be-processed image using a target recognition model to obtain a first recognition result. Among them, the first recognition result includes the position information of the first obstacle.

[0133] In this embodiment, after obtaining the image to be processed, the trained recognition model, i.e., the target recognition model, is used to perform image recognition on the image to be processed to obtain the relevant information of the obstacle in the image to be processed, i.e., the first obstacle, and use it as the first recognition result.

[0134] Among them, the first recognition result includes the position information of the first obstacle. The position information of the first obstacle includes the size information determined according to the set of edge pixel points of the first obstacle and / or the position information of the rectangular area corresponding to the first obstacle.

[0135] Among them, the first obstacle can be a low obstacle or other types of obstacles, and it is not restricted here. Optionally, the process of determining the position information of the rectangular area corresponding to the first obstacle can be: when performing image recognition on the image to be processed, it is necessary to draw the circumscribed two-dimensional rectangular frame of the first obstacle on the image to be processed, so as to describe the first obstacle by using the two-dimensional rectangular frame, that is, the position information of the two-dimensional rectangular frame corresponding to the first obstacle, i.e., the rectangular area, can be used as the position information of the rectangular area corresponding to the first obstacle.

[0136] Among them, the position information of the rectangular area corresponding to the first obstacle can be the position information of the pixel points on the two-dimensional rectangular frame corresponding to the first obstacle. Of course, it can also be other pixel point information related to the rectangular area, i.e., the two-dimensional rectangular frame. For example, the position information of the pixel points at the center of the rectangular area is not restricted here.

[0137] Optionally, when the first obstacle is a low obstacle, the position information of the first obstacle can also include the size information determined according to the set of edge pixel points of the first obstacle, i.e., the size information of the first obstacle. Specifically, when performing image recognition on the image to be processed, it is also possible to recognize the boundary of the first obstacle on the image to be processed, i.e., the pixel points on the edge contour, to obtain the set of edge pixel points, that is, the set of edge pixel points is composed of at least one edge pixel point, i.e., the pixel points on the edge contour. Correspondingly, the size information of the first obstacle can include the position information of the edge pixel points, or can also include the specific size of the first obstacle determined according to the edge pixel points, i.e., the size, which is not restricted here.

[0138] In this embodiment, when determining the size information of the low obstacle, i.e., the first obstacle, it is determined according to the set of pixel points on the actual edge contour of the low obstacle, so that the determined size information is more consistent with the size of the actual obstacle, ensuring the accuracy of the size information.

[0139] Optionally, the target recognition model is a model obtained after training based on relevant learning algorithms (e.g., machine learning algorithms, deep learning algorithms, etc.), which can accurately recognize images to identify the first obstacle (e.g., low obstacle) on the image and the position information of the first obstacle, etc. For example, the target recognition model is trained based on the deep learning algorithm of HRNet.

[0140] Step S203: Obtain the current position information of the second obstacle in the area to be processed. Herein, the second obstacle is an obstacle determined within a preset time period before the current moment.

[0141] In this embodiment, in order to improve the accuracy of obstacle determination, the obstacles in the area corresponding to the image to be processed cannot be determined only based on the first obstacle determined from the image to be processed. Historical obstacles, i.e., the second obstacles, can also be utilized. Then, the current position information of the second obstacle is obtained for determining the target obstacle corresponding to the image to be processed by using the current position information of the second obstacle.

[0142] Herein, the second obstacle refers to an obstacle determined within a preset time period before the current moment, that is, the obstacle recognized in the area to be processed. Specifically, the second obstacle can be the target obstacle corresponding to the previous frame image of the image to be processed determined based on the obstacle recognition process disclosed in this application.

[0143] It can be understood that the current moment represents the moment when the image to be processed is captured. The current position information of the second obstacle represents the position information of the second obstacle at the current moment.

[0144] Optionally, the obstacle type corresponding to the first obstacle is the same as the obstacle type corresponding to the second obstacle. For example, if the obstacle type corresponding to the first obstacle includes the low obstacle type, then the obstacle type corresponding to the second obstacle also includes the low obstacle type.

[0145] Optionally, the current position information of the second obstacle includes the current position information of the second obstacle corresponding to at least one preset tracking algorithm.

[0146] Optionally, the information type included in the current position information of the second obstacle can be the same as the information type included in the position information of the first obstacle, that is, the current position information of the second obstacle includes the current position information of the rectangular area corresponding to the second obstacle and / or the size information determined according to the set of edge pixel points of the second obstacle.

[0147] Step S204: Determine the target obstacle according to the position information of the first obstacle and the current position information of the second obstacle.

[0148] In this embodiment, after obtaining the position information of the first obstacle and the current position information of the second obstacle, based on the position information of the first obstacle and the current position information of the second obstacle, the first obstacle and the second obstacle are fused (that is, the corresponding relationship between the first obstacle and the second obstacle is determined for updating the second obstacle by using the corresponding relationship and the first obstacle), and the obstacles in the area to be processed, that is, the target obstacles corresponding to the image to be processed, are obtained, so as to achieve accurate detection of the target obstacles.

[0149] In addition, optionally, after obtaining the target obstacle corresponding to the image to be processed, the target obstacle can be used as a new historical obstacle, that is, a new second obstacle, so that when determining the target obstacle corresponding to the next frame image of the image to be processed, the new second obstacle can be used for determination.

[0150] In this embodiment, when determining the size information of the obstacle, it is determined according to the set of pixel points on the actual edge contour of the obstacle, so that the determined size information is more consistent with the size of the actual obstacle, ensuring the accuracy rate of the size information, and thus enabling better driving planning according to the size information, ensuring the stability and safety of vehicle driving, and ensuring the obstacle avoidance ability of the vehicle.

[0151] In this embodiment, since the obstacle has a certain continuity, that is, it appears in multiple continuously captured images, the obstacle determined at the current moment can be supplemented by using the obstacle determined before the current moment, so as to effectively avoid missing obstacles and improve the accuracy rate of obstacle determination.

[0152] In this embodiment, compared with being restricted by the installation height or algorithm accuracy when identifying low obstacles through devices such as lidar, millimeter wave radar, or ultrasonic radar, in this application, images are collected by a vision sensor, that is, a camera, and a large amount of redundant texture information can be obtained by using the images, so that low obstacles that cannot be detected by lidar navigation technology can be better detected, realizing accurate perception of low obstacles around the vehicle and ensuring the safety of autonomous driving.

[0153] As described above, when the to-be-processed image of the to-be-processed area captured by the camera is obtained, it indicates that the obstacles in the to-be-processed area need to be recognized. Then, the target recognition model is used to perform image recognition on the to-be-processed image to obtain the first recognition result, that is, to determine the position information of the first obstacle in the to-be-processed image. After determining the first obstacle, the position information of the obstacles determined within the preset time period before the current moment, that is, the second obstacle, and the position information of the first obstacle are comprehensively used to determine the obstacle corresponding to the to-be-processed image, that is, the target obstacle is comprehensively determined according to the second obstacle and the first obstacle, rather than directly taking the first obstacle as the obstacle corresponding to the to-be-processed area, that is, the target obstacle is not determined only based on a single-frame image, which can improve the accuracy of obstacle recognition.

[0154] As Figure 3 shown, Figure 3 FIG. is a flowchart of another obstacle recognition method shown according to an exemplary embodiment of the present specification. In this embodiment, Figure 2 on the basis of the embodiment, when determining the target obstacle, it is determined according to the association result between the first obstacle and the second obstacle. The following will describe this process in detail with a specific example. As Figure 3 shown, the method includes the following steps:

[0155] Step S301: Obtain the to-be-processed image of the to-be-processed area captured by the camera.

[0156] Step S302: Use the target recognition model to perform image recognition on the to-be-processed image to obtain the first recognition result. The first recognition result includes the position information of the first obstacle.

[0157] In this embodiment, the implementation processes of steps S301 - S302 are similar to those of steps S201 - S202 in the Figure 2 embodiment, and will not be elaborated herein.

[0158] Step S303: Obtain the current position information of the second obstacle in the to-be-processed area. The second obstacle is the obstacle determined within the preset time period before the current moment.

[0159] In this embodiment, the current position information of the second obstacle is obtained by predicting the position information at the previous moment of the current moment, that is, the historical position information, using a preset tracking algorithm. The specific process is as follows: Obtain the historical position information of the second obstacle. For each preset tracking algorithm, perform position prediction on the second obstacle according to the historical position information of the second obstacle and the preset tracking algorithm to obtain the current position information of the second obstacle corresponding to the preset tracking algorithm.

[0160] Among them, the preset tracking algorithm can be selected according to actual requirements. For example, tracking algorithms such as the Kalman filter algorithm and the kernel correlation filtering algorithm. The number of preset tracking algorithms is at least one.

[0161] For example, the preset tracking algorithms include the Kalman filter algorithm and the kernel correlation filtering algorithm, and the second obstacle includes obstacle 1. The position information of obstacle 1 at time 1 is position information 1. According to the Kalman filter algorithm and position information 1, the position of obstacle 1 at the next moment of time 1, that is, the current moment, is predicted to obtain the current position information of obstacle 1 corresponding to the Kalman filter algorithm. And according to the kernel correlation filtering algorithm and position information 1, the position of obstacle 1 at the next moment of time 1, that is, the current moment, is predicted to obtain the current position information of obstacle 1 corresponding to the kernel correlation filtering algorithm.

[0162] Among them, the Kalman filter algorithm calculates the motion state of the tracking target, that is, the second obstacle (for example, the change rate of size and position coordinates, acceleration, angular velocity, etc.). Based on the difference between the current moment and the moment when the second obstacle is in a certain state, that is, the moment for position prediction, the change value of the motion state of the second obstacle is obtained, so as to predict the state of the second obstacle at the current moment.

[0163] Among them, the correlation filtering algorithm calculates the features of the image of the second obstacle (for example, gray-scale features, histogram of oriented gradients features, etc.), searches on the image corresponding to the current moment, and obtains the region that is most similar to the image features of the second obstacle. This region is the predicted tracking result of the second obstacle at the current moment.

[0164] Step S304: Determine the association matrix between the second obstacle and the first obstacle according to the current position information of the second obstacle and the position information of the first obstacle.

[0165] In this embodiment, after obtaining the current position information of the second obstacle and the position information of the first obstacle, an association matrix between the second obstacle and the first obstacle is constructed based on the current position information of all second obstacles and the position information of all first obstacles. The association matrix, that is, the obstacles involved in the association matrix include all second obstacles and all first obstacles, for using the association matrix to determine the corresponding relationship between the first obstacle and the second obstacle.

[0166] Among them, the association matrix can represent the association relationship between the first obstacle and the second obstacle. For example, the association relationship is represented by an association score. The higher the association score, the greater the possibility that the two are the same object. The association score can be calculated according to the distance between the first obstacle and the second obstacle or the intersection over union between the regions of the first obstacle and the second obstacle. Here, it is not limited thereto.

[0167] Optionally, when determining the association matrix between the second obstacle and the first obstacle, other methods can also be used for determination. For example, obtaining the texture feature information of the second obstacle and the texture feature information of the first obstacle, and constructing the association matrix between the second obstacle and the first obstacle based on the texture feature information of the second obstacle and the texture feature information of the first obstacle. The present application does not limit the method for determining the association matrix.

[0168] Step S305: Obtain the target association result between the second obstacle and the first obstacle according to the association matrix.

[0169] In this embodiment, after obtaining the association matrix between the second obstacle and the first obstacle, based on the optimal matching algorithm (for example, the Hungarian algorithm), the association matrix is matched to obtain the corresponding relationship between the second obstacle and the first obstacle, that is, the corresponding association result, and the target association result between the second obstacle and the first obstacle is determined according to this association result.

[0170] Optionally, when the number of preset tracking algorithms is one, the corresponding association matrix is determined according to the position information of each first obstacle and the current position information of each second obstacle corresponding to the preset tracking algorithm, and based on the optimal matching algorithm, the association matrix is matched to obtain the corresponding association result, that is, the association result corresponding to the preset tracking algorithm, and the association result corresponding to the preset tracking algorithm is directly used as the target association result.

[0171] Optionally, when the number of preset tracking algorithms is multiple, it may be necessary to determine the association matrix corresponding to each preset tracking algorithm, so that the initial association result corresponding to each preset tracking algorithm is determined according to the association matrix corresponding to each preset tracking algorithm, and the final association result, that is, the target association result, is determined according to the association result corresponding to each preset tracking algorithm. The specific process is as follows:

[0172] Obtain the priority corresponding to each preset tracking algorithm.

[0173] Select the target tracking algorithm from all the preset tracking algorithms according to the descending order of the priorities, and obtain the current position information of the second obstacle corresponding to the target tracking algorithm.

[0174] Determine the association matrix corresponding to the target tracking algorithm according to the current position information of the second obstacle corresponding to the target tracking algorithm and the position information of the first obstacle.

[0175] Determine the initial association result between the second obstacle and the first obstacle corresponding to the target tracking algorithm according to the association matrix corresponding to the target tracking algorithm.

[0176] If there are failures in associating with the first obstacle and the second obstacle in the initial association results, obtain the number of determined association matrices, and determine whether the number of determined association matrices is equal to the number of preset tracking algorithms.

[0177] If it is equal to the number of preset tracking algorithms, perform a merging process on all the initial association results to obtain the target association results.

[0178] If it is not equal to the number of preset tracking algorithms, continue to select the target tracking algorithm from all the preset tracking algorithms according to the priority from high to low, and determine the association matrix corresponding to the target tracking algorithm based on the current position information of the remaining second obstacles and the position information of the remaining first obstacles corresponding to the target tracking algorithm. Among them, the initial association result corresponding to the remaining second obstacle is the failure in associating with the second obstacle, and the initial association result corresponding to the remaining first obstacle is the failure in associating with the first obstacle.

[0179] Specifically, sort according to the priority of the preset tracking algorithms from high to low. First, select the preset tracking algorithm with a high priority and use it as the target tracking algorithm. Determine an association matrix based on the current position information of the second obstacle corresponding to this target tracking algorithm (i.e., the current position information obtained by predicting the position of this second obstacle through this target tracking algorithm) and the position information of the first obstacle, and use it as the association matrix corresponding to this target tracking algorithm. Based on the optimal matching algorithm, match the association matrix corresponding to this target tracking algorithm to obtain the association result between the second obstacle and the first obstacle, and determine it as the initial association result between the second obstacle and the first obstacle corresponding to this target tracking algorithm, that is, the initial association result corresponding to this target tracking algorithm. Determine whether there are failures in associating with the first obstacle and the second obstacle in the initial association result corresponding to this target tracking algorithm, that is, determine whether there are second obstacles that are not corresponding to any first obstacle and first obstacles that are not corresponding to any second obstacle.

[0180] When it is determined that there is a second obstacle that does not correspond to any first obstacle, that is, the remaining second obstacles and the first obstacles that do not correspond to any second obstacle, that is, the remaining first obstacles, then continue to use the preset tracking algorithm with a lower priority to determine the correspondence between the remaining second obstacles and the remaining first obstacles. That is, select from the preset tracking algorithms the one that is adjacent to the target tracking algorithm and has a lower priority than the priority corresponding to the target tracking algorithm, and determine it as the new target tracking algorithm. Then, based on the current position information of the remaining second obstacles corresponding to the new target tracking algorithm, determine the association matrix between the remaining second obstacles and the remaining first obstacles. Thus, based on this association matrix, obtain the association result between the remaining second obstacles and the remaining first obstacles corresponding to the new target tracking algorithm, that is, the initial association result between the second obstacles and the first obstacles corresponding to the new target tracking algorithm, that is, the initial association result corresponding to the new target tracking algorithm. Correspondingly, the initial association results corresponding to each preset tracking algorithm can be obtained.

[0181] In addition, optionally, when determining the initial association result where there is no failure in associating the first obstacles and / or the second obstacles, it indicates that all the first obstacles and / or all the second obstacles have been corresponded, and there is no need to continue to determine the initial association results corresponding to other preset tracking algorithms. For example, the preset tracking algorithms include Algorithm 1, Algorithm 2, and Algorithm 3. Algorithm 1 has the highest priority, followed by Algorithm 2, and Algorithm 3 has the lowest priority. After obtaining the initial association result corresponding to Algorithm 1, the initial association results corresponding to Obstacle 1 and Obstacle 2 are failures in associating the first obstacles, and the initial association result corresponding to Obstacle 3 is a failure in associating the second obstacle, indicating that no second obstacle corresponds to Obstacle 1, no second obstacle corresponds to Obstacle 2, and no first obstacle corresponds to Obstacle 3. Then, Obstacle 1 and 2 are both remaining first obstacles, and Obstacle 3 is the remaining second obstacle. Based on the current position information of Obstacle 3 corresponding to Algorithm 2 and the position information of Obstacle 1 and 2, determine that the initial association result corresponding to Algorithm 2 includes a successful association between the first obstacle and the second obstacle and a failure in associating the first obstacle. The first obstacle corresponding to the initial association result of the successful association between the first obstacle and the second obstacle is Obstacle 1, and the second obstacle corresponding to the initial association result of the successful association between the first obstacle and the second obstacle is Obstacle 3, that is, it indicates that Obstacle 1 and 3 are successfully associated. The first obstacle corresponding to the initial association result of the failure in associating the first obstacle is Obstacle 2. Since there is no uncorresponded second obstacle, that is, there is no remaining second obstacle, there is no need to determine the initial association result corresponding to Algorithm 3.

[0182] Optionally, when merging all the obtained initial association results, the initial association results corresponding to the preset tracking algorithm with a lower priority are used to update the initial association results corresponding to the preset tracking algorithm with a higher priority. That is, for each first obstacle, when there is an initial association result in which a second obstacle is successfully associated with the first obstacle among all the initial association results corresponding to the first obstacle, it indicates that when determining the association result according to the preset tracking algorithm corresponding to the initial association result, there is a second obstacle corresponding to the first obstacle. Then, the initial association result in which the second obstacle is successfully associated with the first obstacle is used as the target association result corresponding to the first obstacle. For each second obstacle, when there is an initial association result in which a second obstacle is successfully associated with the first obstacle among all the initial association results corresponding to the second obstacle, it indicates that when determining the association result according to the preset tracking algorithm corresponding to the initial association result, there is a first obstacle corresponding to the second obstacle. Then, the initial association result in which the second obstacle is successfully associated with the first obstacle is used as the target association result corresponding to the second obstacle.

[0183] Among them, the setting of the priority corresponding to the preset tracking algorithm can be set by relevant personnel according to actual needs. For example, when the requirement for determining the speed of the obstacle is high, the priority of the tracking algorithm that can determine the position tracking result faster can be set higher; when the requirement for determining the accuracy of the obstacle is high, the priority of the tracking algorithm that can determine the position tracking result more accurately can be set higher.

[0184] Step S306: Update the second obstacle according to the target association result, and determine the low obstacle according to the updated second obstacle.

[0185] In this embodiment, the second obstacle is updated by using the corresponding relationship between the second obstacle and the first obstacle, that is, the first obstacle is updated to obtain the updated second obstacle. The fusion between the second obstacle and the first obstacle is realized, and the fused second obstacle, that is, the updated second obstacle, is used as the target obstacle, that is, the low obstacle.

[0186] Optionally, the current position information of the updated second obstacle is the current position information of the target obstacle.

[0187] In this embodiment, optionally, when updating the second obstacle according to the target association result, when it is determined that the target association result is that the second obstacle is successfully associated with the first obstacle, the current position information of the second obstacle corresponding to the target association result is updated according to the position information of the first obstacle corresponding to the target association result. When it is determined that the target association result is that the association of the first obstacle fails, the first obstacle corresponding to the target association result is used as the new second obstacle. When it is determined that the target association result is that the association of the second obstacle fails, the second obstacle corresponding to the target association result is updated.

[0188] Specifically, when the target association result is that the second obstacle is successfully associated with the first obstacle, it indicates that the second obstacle still exists on the image to be processed, and the second obstacle and the first obstacle belong to the same obstacle. Then, the position information of the first obstacle, that is, the actual position information of the obstacle, can be used to update the current position information of the second obstacle, that is, the position information of the first obstacle information is used as the current position information of the second obstacle, so that the current position information of the updated second obstacle is more accurate, and the accurate identification of the position of the target obstacle is realized. For example, when the first obstacle includes obstacle a and the second obstacle includes obstacle A, when it is determined that obstacle a and obstacle A are successfully associated, it means that obstacle A and obstacle a are the same obstacle, and then the position information of obstacle a is used as the current position information of obstacle A to update the position information of obstacle A.

[0189] Specifically, when the target association result is that the association of the first obstacle fails, it indicates that the first obstacle is not corresponding to any second obstacle, and the first obstacle may be a newly added obstacle, that is, a new obstacle enters the area corresponding to the captured image. Then, the first obstacle is added to the second obstacles, that is, the first obstacle is used as the new second obstacle for subsequent tracking, so as to perform subsequent obstacle recognition. Correspondingly, the position information of the first obstacle is the current position information of the new obstacle, and the size information of the first obstacle is the current size information of the new obstacle.

[0190] Specifically, when the target association result is that the association of the second obstacle fails, it indicates that the second obstacle is not corresponding to any first obstacle, and the second obstacle does not exist on the image to be processed. The reasons include that the second obstacle actually exists in the area corresponding to the image to be processed but fails to be successfully recognized at the current moment, or the second obstacle does not actually exist in this area, that is, reasons such as leaving the area corresponding to the captured image. Therefore, in order to ensure the stability of the tracking result, it is necessary to further determine the reason why the second obstacle does not appear on the image to be processed, so as to determine whether to delete the second obstacle based on this reason, that is, to update the second obstacle.

[0191] Optionally, when updating the second obstacle corresponding to the target association result, obtain the status information of the second obstacle corresponding to the target association result. Determine whether the status information meets the preset deletion condition.

[0192] When it is determined that the status information meets the preset deletion condition, delete the second obstacle corresponding to the target association result. When it is determined that the status information does not meet the preset deletion condition, update the status information.

[0193] Specifically, when the target association result is a failure to associate with the second obstacle, obtain the status information of the second obstacle, and determine whether the status information meets the preset deletion condition to determine whether to delete the second obstacle. When it is determined that the status information meets the preset deletion condition, it indicates that it is determined to delete the second obstacle, that is, it indicates that the second obstacle does not actually exist in the area corresponding to the image to be processed, so directly delete the second obstacle and stop tracking the second obstacle. When it is determined that the status information does not meet the preset deletion condition, it indicates that it is determined not to delete the second obstacle, that is, it indicates that the second obstacle may exist in the area corresponding to the image to be processed, but the second obstacle fails to be successfully recognized, so update the status information corresponding to the second obstacle.

[0194] Optionally, the status information corresponding to the second obstacle includes a recognition failure ratio, which represents the ratio of the number of images in which the second obstacle is not detected in a preset number of images before the image to be processed.

[0195] Taking a specific application scenario as an example, the preset number is 4. Obtain the four images taken before the current moment, that is, the image to be processed, namely Image 1, Image 2, Image 3, and Image 4. The second obstacle is recognized in Image 1, the second obstacle is recognized in Image 2, the second obstacle is recognized in Image 3, and the second obstacle is not recognized in Image 4. Then the recognition failure ratio corresponding to the second obstacle is 1 / 4.

[0196] Correspondingly, when the status information corresponding to the second obstacle includes a recognition failure ratio, determining whether the status information meets the preset deletion condition includes: determining whether the recognition failure ratio is greater than a preset ratio threshold. If it is greater than the preset ratio threshold, it is determined that the status information meets the preset deletion condition. If it is less than or equal to the preset ratio threshold, it is determined that the status information does not meet the preset deletion condition.

[0197] Specifically, after obtaining the recognition failure ratio corresponding to the second obstacle, it is determined whether the recognition failure ratio is greater than a preset ratio threshold. When it is determined that the recognition failure ratio is greater than the preset ratio threshold, it indicates that the number of failures in recognizing the second obstacle is large, and the second obstacle is actually no longer in the area corresponding to the image to be processed. The second obstacle can be deleted from the second obstacle set, that is, the historical obstacle set, and then it is determined that the status information corresponding to the second obstacle meets the preset deletion condition. When it is determined that the recognition failure ratio is less than or equal to the preset ratio threshold, it indicates that the number of failures in recognizing the second obstacle is small, and the second obstacle may still be in the area corresponding to the image to be processed, and only the recognition ratio needs to be updated.

[0198] Continuing with the above application scenario, the preset ratio threshold is 1 / 2, and the recognition failure ratio corresponding to the second obstacle is 1 / 4, which is less than the preset ratio threshold. Then, the recognition failure ratio is updated. That is, the image to be processed and the 3 images before the image to be processed are used as the preset number of images required to determine the updated recognition failure. Since the second obstacle is not recognized in the image to be processed, the updated recognition failure ratio is 2 / 4, that is, 1 / 2.

[0199] In any embodiment, optionally, the electronic map corresponding to the area to be processed is updated according to the target obstacle. The target obstacle is an obstacle with a height less than a preset threshold.

[0200] Specifically, after obtaining the target obstacle, the electronic map can also be updated using the target obstacle, so that the updated electronic device matches the actual environment better and has higher accuracy, and thus a more stable and comfortable driving plan can be made for the vehicle according to the updated electronic map.

[0201] Optionally, when updating the electronic map, the electronic map needs to be mapped to the image to be processed, that is, the positioning information of the vehicle when taking the image to be processed is obtained, that is, the vehicle positioning information corresponding to the current moment, and based on the vehicle positioning information, the electronic map corresponding to the area around the vehicle is obtained, that is, the electronic map corresponding to the area to be processed. Each map point in the electronic map corresponding to the area to be processed is projected onto the image to be processed to determine whether each map point is a target obstacle.

[0202] Among them, when projecting the map point onto the image to be processed, the projection can be performed according to the existing projection process. For example, after determining the relative position relationship between the map point and the current vehicle, combined with the internal and external parameters calibrated by the camera on the vehicle, that is, the on-vehicle camera, and referring to the pinhole imaging principle of the camera, the map point is projected from the three-dimensional space onto the two-dimensional plane of the image.

[0203] Among them, when determining whether a map point is a target obstacle, if the map point projected on the image to be processed coincides with a target obstacle, then the map point is determined to be a target obstacle. For example, a low obstacle.

[0204] In this embodiment, according to the current position information of the second obstacle and the position information of the first obstacle, the association degree between the second obstacle and the first obstacle, that is, the association matrix, is determined, and the association result between the second obstacle and the first obstacle, that is, the corresponding relationship, is determined by using the association matrix, so as to use the corresponding relationship to determine the target obstacle in the area to be processed. That is, when determining the target obstacle in the area to be processed, the historical obstacle, that is, the second obstacle, is fused to achieve accurate and stable recognition of the target obstacle and ensure the accuracy rate of obstacle recognition.

[0205] Such as Figure 4 shown, Figure 4 FIG. is a flowchart of a driving method according to an exemplary embodiment of the present specification. The execution subject corresponding to the method is a vehicle-mounted terminal, and the method includes the following steps:

[0206] Step S401, obtain an updated electronic map.

[0207] Step S401, determine a driving route based on the updated electronic map so that the vehicle can perform autonomous driving according to the driving route.

[0208] In this embodiment, when performing autonomous driving, an updated electronic map can be obtained and the updated electronic map can be loaded to use the updated electronic map, that is, an electronic map that better matches the actual environment, for driving route planning, that is, to determine the driving route of the vehicle, so that the vehicle can perform autonomous driving according to the driving route, ensuring the safety and stability of autonomous driving and correspondingly improving the comfort level.

[0209] Among them, the target obstacles (such as low obstacles) in the updated electronic map are determined according to the obstacle recognition process described in the above embodiments, and will not be elaborated here.

[0210] Among them, the process of formulating a driving route using the updated electronic map is similar to the existing process of formulating a driving route using a map. For example, it is formulated using the position and size information of obstacles in the electronic map, and will not be elaborated here.

[0211] In this embodiment, since the updated electronic map better matches the actual environment, when the vehicle travels according to the driving route determined based on the updated electronic map, it can effectively avoid target obstacles, ensuring the safety and stability of driving.

[0212] It should be emphasized that in the prior art, an image is generally segmented into a drivable area and a non-drivable area (which may include a background area, an area where low obstacles are located, etc.). Based on the drivable area, a driving plan is made for the vehicle, that is, the vehicle is restricted to drive within the drivable area. Therefore, to a certain extent, the driving range of the vehicle is reduced, and when the division of the drivable area is poor, it will greatly affect the driving range of the vehicle and cannot better make a driving plan for the vehicle. In contrast, the present application only identifies the target obstacles in the image and makes a driving rule based on the area where the target obstacles are located, so that the vehicle can drive in the area other than the area where the target obstacles are located, ensuring the driving range of the vehicle, and thus can better make a driving plan for the vehicle.

[0213] Corresponding to the embodiments of the foregoing method, this specification also provides embodiments of an obstacle recognition device and a terminal to which it is applied.

[0214] The embodiments of the obstacle recognition device in this specification can be applied to a computer device, such as a server or a terminal device. The device embodiments can be implemented by software, or by hardware or a combination of software and hardware. Taking software implementation as an example, as a logically meaningful device, it is formed by the processor in the file processing reading the corresponding computer program instructions in the non-volatile memory into the memory for operation. From the hardware level, as Figure 5 shown, is a hardware structure diagram of the computer device where the obstacle recognition device in the embodiments of this specification is located. In addition to Figure 5 the processor 510, the memory 530, the network interface 520, and the non-volatile memory 540 shown, the server or electronic device where the obstacle recognition device 531 is located in the embodiments usually also includes other hardware according to the actual functions of the computer device, which will not be elaborated here.

[0215] Corresponding to the embodiments of the foregoing method, this specification also provides embodiments of a driving device and a terminal to which it is applied.

[0216] The embodiments of the driving device in this specification can be applied to an in-vehicle terminal. The device embodiments can be implemented by software, or by hardware or a combination of software and hardware. Taking software implementation as an example, as a logically meaningful device, it is formed by the processor where it is located reading the corresponding computer program instructions in the non-volatile memory into the memory for operation. From the hardware level, as Figure 6 shown, is a hardware structure diagram of the computer device where the driving device in the embodiments of this specification is located. In addition to Figure 6In addition to the processor 610, memory 630, network interface 620, and non-volatile memory 640 shown, in an embodiment, the in-vehicle terminal where the driving device 631 is located usually may further include other hardware according to the actual functions of the in-vehicle terminal, which will not be elaborated herein.

[0217] As Figure 7 shown, Figure 7 is a block diagram of an obstacle recognition device shown in accordance with an exemplary embodiment of the present specification. The device includes:

[0218] An image acquisition module 710, configured to acquire a to-be-processed image of a to-be-processed area captured by a camera.

[0219] An image processing module 720, configured to perform image recognition on the to-be-processed image by using a target recognition model to obtain a first recognition result. Wherein, the first recognition result includes position information of a first obstacle.

[0220] An information acquisition module 730, configured to acquire current position information of a second obstacle in the to-be-processed area. Wherein, the second obstacle is an obstacle determined within a preset time period before the current moment.

[0221] An obstacle recognition module 740, configured to determine a target obstacle according to the position information of the first obstacle and the current position information of the second obstacle.

[0222] Optionally, the size information of the first obstacle includes size information determined according to a set of edge pixel points of the first obstacle and / or position information of a rectangular area corresponding to the first obstacle.

[0223] In another embodiment of the present application, based on the above Figure 6 embodiment, the target obstacle includes a low obstacle;

[0224] The obstacle recognition module 740 includes:

[0225] An association matrix determination unit, configured to determine an association matrix between the second obstacle and the first obstacle according to the current position information of the second obstacle and the position information of the first obstacle.

[0226] An association result determination unit, configured to obtain a target association result between the second obstacle and the first obstacle according to the association matrix.

[0227] An obstacle determination unit, configured to perform an update process on the second obstacle according to the target association result, and determine a low obstacle according to the updated second obstacle.

[0228] In this embodiment, optionally, the obstacle determination unit is specifically configured to:

[0229] When it is determined that the target association result is that the second obstacle is successfully associated with the first obstacle, update the current position information of the second obstacle corresponding to the target association result according to the position information of the first obstacle corresponding to the target association result.

[0230] When it is determined that the target association result is that the first obstacle is associated unsuccessfully, use the first obstacle corresponding to the target association result as the new second obstacle.

[0231] When it is determined that the target association result is that the second obstacle is associated unsuccessfully, perform an update process on the second obstacle corresponding to the target association result.

[0232] In this embodiment, optionally, the obstacle determination unit is specifically configured to:

[0233] Obtain the status information of the second obstacle corresponding to the target association result.

[0234] Determine whether the status information meets a preset deletion condition.

[0235] When it is determined that the status information meets the preset deletion condition, delete the second obstacle corresponding to the target association result.

[0236] When it is determined that the status information does not meet the preset deletion condition, update the status information.

[0237] In this embodiment, optionally, the obstacle determination unit is specifically configured to:

[0238] Determine whether the recognition failure ratio is greater than a preset ratio threshold.

[0239] If it is greater than the preset ratio threshold, determine that the status information meets the preset deletion condition.

[0240] If it is less than or equal to the preset ratio threshold, determine that the status information does not meet the preset deletion condition.

[0241] Optionally, the current position information of the second obstacle includes the current position information of the second obstacle corresponding to at least one preset tracking algorithm.

[0242] In this embodiment, optionally, the association matrix determination unit is specifically configured to:

[0243] Obtain the priority corresponding to each preset tracking algorithm.

[0244] Select a target tracking algorithm from all the preset tracking algorithms according to the sorting of the priorities from high to low, and obtain the current position information of the second obstacle corresponding to the target tracking algorithm.

[0245] Determine the association matrix corresponding to the target tracking algorithm according to the current position information of the second obstacle corresponding to the target tracking algorithm and the position information of the first obstacle.

[0246] In this embodiment, optionally, the association result determination unit is specifically configured to:

[0247] Determine the initial association result between the second obstacle corresponding to the target tracking algorithm and the first obstacle according to the association matrix corresponding to the target tracking algorithm.

[0248] If there are cases where the first obstacle fails to be associated and the second obstacle fails to be associated in the initial association result, obtain the number of determined association matrices, and determine whether the number of determined association matrices is equal to the number of preset tracking algorithms.

[0249] If it is equal to the number of preset tracking algorithms, perform a merging process on all the initial association results to obtain the target association result.

[0250] If it is not equal to the number of preset tracking algorithms, continue to select the target tracking algorithm from all the preset tracking algorithms according to the priority from high to low, and determine the association matrix corresponding to the target tracking algorithm according to the current position information of the remaining second obstacles and the position information of the remaining first obstacles corresponding to the target tracking algorithm. Among them, the initial association result corresponding to the remaining second obstacles is that the second obstacle fails to be associated, and the initial association result corresponding to the remaining first obstacles is that the first obstacle fails to be associated.

[0251] In any embodiment, optionally, the information acquisition module 730 is further configured to:

[0252] Before obtaining the current position information of the second obstacle, obtain the historical position information of the second obstacle.

[0253] For each preset tracking algorithm, perform position prediction on the second obstacle according to the historical position information of the second obstacle and the preset tracking algorithm to obtain the current position information of the second obstacle corresponding to the preset tracking algorithm.

[0254] In any embodiment, optionally, the obstacle recognition device further includes:

[0255] A map update module, configured to update the electronic map corresponding to the area to be processed according to the target obstacle. The target obstacle is an obstacle with a height less than a preset threshold.

[0256] The implementation processes of the functions and roles of each module in the above device are specifically described in detail in the implementation processes of the corresponding steps in the above method, and will not be elaborated here.

[0257] Correspondingly, this specification also provides a computer device, which includes a processor; a memory for storing instructions executable by the processor; wherein, the processor is configured to: obtain a to-be-processed image of a to-be-processed area captured by a camera. Perform image recognition on the to-be-processed image using an object recognition model to obtain a first recognition result. The first recognition result includes position information of a first obstacle. Obtain current position information of a second obstacle in the to-be-processed area. The second obstacle is an obstacle determined within a preset time period before the current moment. Determine a target obstacle based on the position information of the first obstacle and the current position information of the second obstacle.

[0258] As Figure 8 shown, Figure 8 is a block diagram of a driving device shown in this specification according to an exemplary embodiment, which is applied to an in-vehicle terminal. The device includes:

[0259] A map acquisition module 810, configured to acquire an updated electronic map. The updated electronic map is obtained by the method according to the first aspect and various possible designs of the first aspect.

[0260] An automatic driving module 820, configured to determine a driving route based on the updated electronic map, so that the vehicle performs automatic driving according to the driving route.

[0261] For the implementation processes of the functions and roles of each module in the above device, refer to the implementation processes of the corresponding steps in the above method for details, which will not be elaborated here.

[0262] Correspondingly, this specification also provides an in-vehicle terminal, which includes a processor; a memory for storing instructions executable by the processor; wherein, the processor is configured to: acquire an updated electronic map. The updated electronic map is obtained by the method according to the first aspect and various possible designs of the first aspect. Determine a driving route based on the updated electronic map, so that the vehicle performs automatic driving according to the driving route.

[0263] An embodiment of the present application provides a computer-readable storage medium, in which computer program code is stored, and when the computer program code is executed by a processor, the above method steps are implemented.

[0264] An embodiment of the present application also provides a computer program product, including a computer program, and when the computer program is executed by a processor, the above method steps are implemented.

[0265] For the apparatus embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to the descriptions of the method embodiments. The apparatus embodiments described above are merely illustrative. The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed to multiple network modules. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution in this specification. Those of ordinary skill in the art can understand and implement it without creative work.

[0266] The above describes specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in a different order than in the embodiments and still achieve the desired results. Additionally, the processes depicted in the figures do not necessarily require the particular order or sequential order shown to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0267] Those skilled in the art will readily conceive of other embodiments of this specification after considering the specification and practicing the invention herein. This specification is intended to cover any variations, uses, or adaptations of this specification, which follow the general principles of this specification and include common general knowledge or conventional technical means in the technical field not claimed in this application. The specification and embodiments are only regarded as exemplary, and the true scope and spirit of this specification are pointed out by the following claims.

[0268] It should be understood that this specification is not limited to the exact structures described above and shown in the figures, and various modifications and changes can be made without departing from its scope. The scope of this specification is only limited by the appended claims.

[0269] The above is only the preferred embodiment of this specification and is not intended to limit this specification. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of this specification shall be included within the scope of protection of this specification.

Claims

1. An obstacle recognition method, characterized in that, Including: Obtaining a to-be-processed image of a to-be-processed area captured by a camera; Performing image recognition on the to-be-processed image by using a target recognition model to obtain a first recognition result; wherein, the first recognition result includes position information of a first obstacle; Obtaining current position information of a second obstacle in the to-be-processed area; wherein, the second obstacle is an obstacle determined within a preset time period before the current moment; Determining a target obstacle according to the position information of the first obstacle and the current position information of the second obstacle, the target obstacle including a low obstacle; Wherein, the determining the target obstacle according to the position information of the first obstacle and the current position information of the second obstacle includes: determining an association matrix between the second obstacle and the first obstacle according to the current position information of the second obstacle and the position information of the first obstacle; obtaining a target association result between the second obstacle and the first obstacle according to the association matrix; performing an update process on the second obstacle according to the target association result, and determining the low obstacle according to the updated second obstacle; Wherein, the current position information of the second obstacle includes current position information of the second obstacle corresponding to at least one preset tracking algorithm; the determining the association matrix between the second obstacle and the first obstacle according to the current position information of the second obstacle and the position information of the first obstacle includes: obtaining a priority corresponding to each preset tracking algorithm; selecting a target tracking algorithm from all the preset tracking algorithms according to the order from high to low of the priorities, and obtaining the current position information of the second obstacle corresponding to the target tracking algorithm; determining the association matrix corresponding to the target tracking algorithm according to the current position information of the second obstacle corresponding to the target tracking algorithm and the position information of the first obstacle.

2. The method according to claim 1, wherein The performing the update process on the second obstacle according to the target association result includes: When determining that the target association result is that the second obstacle is successfully associated with the first obstacle, updating the current position information of the second obstacle corresponding to the target association result according to the position information of the first obstacle corresponding to the target association result; When determining that the target association result is that the first obstacle is associated unsuccessfully, using the first obstacle corresponding to the target association result as a new second obstacle; When determining that the target association result is that the second obstacle is associated unsuccessfully, performing an update process on the second obstacle corresponding to the target association result.

3. The method according to claim 2, wherein The performing the update process on the second obstacle corresponding to the target association result includes: Obtaining the state information of the second obstacle corresponding to the target association result; Judging whether the state information meets a preset deletion condition; When determining that the state information meets the preset deletion condition, deleting the second obstacle corresponding to the target association result; When determining that the state information does not meet the preset deletion condition, updating the state information.

4. The method according to claim 3, characterized in that Wherein, the state information includes a recognition failure ratio, and the judging whether the state information meets the preset deletion condition includes: Determine whether the recognition failure ratio is greater than a preset ratio threshold; If it is greater than the preset ratio threshold, determine that the status information meets the preset deletion condition; If it is less than or equal to the preset ratio threshold, determine that the status information does not meet the preset deletion condition.

5. The method according to claim 1, wherein The obtaining the target association result between the second obstacle and the first obstacle according to the association matrix includes: Determine the initial association result between the second obstacle and the first obstacle corresponding to the target tracking algorithm according to the association matrix corresponding to the target tracking algorithm; If there are cases of the first obstacle association failure and the second obstacle association failure in the initial association result, obtain the number of determined association matrices, and determine whether the number of determined association matrices is equal to the number of the preset tracking algorithms; If it is equal to the number of the preset tracking algorithms, perform a merging process on all the initial association results to obtain the target association result; If it is not equal to the number of the preset tracking algorithms, continue to select the target tracking algorithm from all the preset tracking algorithms in the order of priority from high to low, and determine the association matrix corresponding to the target tracking algorithm according to the current position information of the remaining second obstacles and the position information of the remaining first obstacles corresponding to the target tracking algorithm; wherein, the initial association result corresponding to the remaining second obstacles is the second obstacle association failure, and the initial association result corresponding to the remaining first obstacles is the first obstacle association failure.

6. The method according to claim 1, wherein Before obtaining the current position information of the second obstacle, it further includes: Obtain the historical position information of the second obstacle; For each preset tracking algorithm, predict the position of the second obstacle according to the historical position information of the second obstacle and the preset tracking algorithm to obtain the current position information of the second obstacle corresponding to the preset tracking algorithm.

7. The method according to claim 1, characterized in that, The method further includes: Update the electronic map corresponding to the area to be processed according to the target obstacle; wherein, the target obstacle is an obstacle with a height less than a preset threshold.

8. The method according to any one of claims 1 to 7, characterized in that The position information of the first obstacle includes size information determined according to the set of edge pixel points of the first obstacle and / or the position information of the rectangular area corresponding to the first obstacle.

9. A driving method, characterized in that, Applied to a vehicle, the method includes: Obtain the updated electronic map; wherein, the updated electronic map is obtained according to the method according to any one of claims 1 to 8; Determine the driving route based on the updated electronic map, so that the vehicle performs autonomous driving according to the driving route.

10. An obstacle recognition device, characterized in that, The device includes: An image acquisition module, configured to acquire a to-be-processed image of the area to be processed captured by a camera; An image processing module, configured to perform image recognition on the to-be-processed image by using a target recognition model to obtain a first recognition result; wherein, the first recognition result includes the position information of the first obstacle; An information acquisition module, configured to acquire the current position information of a second obstacle in the area to be processed; wherein, the second obstacle is an obstacle determined within a preset time period before the current moment; An obstacle recognition module, configured to determine a target obstacle according to the position information of the first obstacle and the current position information of the second obstacle, where the target obstacle includes a low obstacle; Wherein, the obstacle recognition module is configured to determine an association matrix between the second obstacle and the first obstacle according to the current position information of the second obstacle and the position information of the first obstacle; obtain a target association result between the second obstacle and the first obstacle according to the association matrix; perform an update process on the second obstacle according to the target association result, and determine the low obstacle according to the updated second obstacle; Wherein, the current position information of the second obstacle includes the current position information of the second obstacle corresponding to at least one preset tracking algorithm; the obstacle recognition module is configured to obtain the priority corresponding to each preset tracking algorithm; select a target tracking algorithm from all the preset tracking algorithms according to the priority from high to low, and obtain the current position information of the second obstacle corresponding to the target tracking algorithm; determine the association matrix corresponding to the target tracking algorithm according to the current position information of the second obstacle corresponding to the target tracking algorithm and the position information of the first obstacle.

11. A driving device, characterized in that, Applied to a vehicle, the device includes: A map acquisition module, configured to acquire an updated electronic map; wherein, the updated electronic map is obtained according to the method according to any one of claims 1 to 8; An automatic driving module, configured to determine a driving route based on the updated electronic map, so that the vehicle performs automatic driving according to the driving route.

12. A computer device, characterized in that, Including a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein when the processor executes the program, the following method is implemented: Obtain a to-be-processed image of a to-be-processed area captured by a camera; Perform image recognition on the to-be-processed image by using a target recognition model to obtain a first recognition result; wherein, the first recognition result includes the position information of a first obstacle; Obtain the current position information of a second obstacle in the to-be-processed area; wherein, the second obstacle is an obstacle determined within a preset time period before the current moment; Determine a target obstacle according to the position information of the first obstacle and the current position information of the second obstacle, where the target obstacle includes a low obstacle; Wherein, determining the target obstacle according to the position information of the first obstacle and the current position information of the second obstacle includes: determining an association matrix between the second obstacle and the first obstacle according to the current position information of the second obstacle and the position information of the first obstacle; obtaining a target association result between the second obstacle and the first obstacle according to the association matrix; performing an update process on the second obstacle according to the target association result, and determining the low obstacle according to the updated second obstacle; Among them, the current position information of the second obstacle includes the current position information of the second obstacle corresponding to at least one preset tracking algorithm; the determining the association matrix between the second obstacle and the first obstacle according to the current position information of the second obstacle and the position information of the first obstacle includes: obtaining the priority corresponding to each preset tracking algorithm; selecting a target tracking algorithm from all the preset tracking algorithms according to the descending order of the priorities, and obtaining the current position information of the second obstacle corresponding to the target tracking algorithm; determining the association matrix corresponding to the target tracking algorithm according to the current position information of the second obstacle corresponding to the target tracking algorithm and the position information of the first obstacle.

13. A vehicle-mounted terminal, characterized in that, It includes a memory, a processor, and a computer program stored on the memory and executable on the processor. Among them, when the processor executes the program, the following method is implemented: Obtain an updated electronic map; among them, the updated electronic map is obtained according to the method according to any one of claims 1 to 8; Determine a driving route based on the updated electronic map and drive according to the driving route.

Images