Obstacle recognition methods, devices, electronic equipment and storage media

Abstract

This invention discloses an obstacle recognition method. The method includes: acquiring images within a defined road area and detecting obstacles to be recognized in the images; when the obstacle to be recognized meets recognition conditions and cannot be recognized based on existing obstacle models in a database, entering an obstacle recognition mode according to the vehicle's current driving mode, and obtaining the recognition result of the obstacle through interaction with a server. This technical solution improves the obstacle recognition capability of the entire interactive system by interacting with a server to recognize the obstacle when it meets recognition conditions and existing obstacle models in the database cannot identify it.

Description

Technical Field

[0001] The present invention relates to the field of driving safety technology, and in particular to an obstacle recognition method, device, electronic device and storage medium. Background Technology

[0002] With the continuous development of driving safety technology, the demand for obstacle recognition during vehicle driving is constantly increasing, requiring vehicles to be able to identify all obstacles encountered during driving.

[0003] In existing technologies, obstacle recognition is mostly achieved by setting up an image processing system in the vehicle. By storing obstacle information in the image processing system, the obstacle information is retrieved when an obstacle is encountered.

[0004] However, current obstacle recognition technologies can only identify obstacles stored in the vehicle's image processing system, failing to guarantee the complete identification of all obstacles encountered on the road. This can lead to traffic accidents and driver injuries or fatalities. Therefore, improving obstacle recognition capabilities is a pressing technical problem that needs to be solved. Summary of the Invention

[0005] This invention provides an obstacle recognition method, device, electronic device, and storage medium to solve the problem that vehicles cannot completely identify all obstacles they may encounter on the road.

[0006] In a first aspect, embodiments of the present invention provide an obstacle recognition method, comprising:

[0007] Acquire images within a defined road area and detect obstacles to be identified in the images;

[0008] When the obstacle to be identified meets the identification conditions but cannot be identified according to the existing obstacle models in the database, the vehicle enters the obstacle identification mode according to the current driving mode of the vehicle, so as to obtain the identification result of the obstacle to be identified through interaction with the server.

[0009] In a second aspect, embodiments of the present invention provide an obstacle recognition device, comprising:

[0010] The obstacle detection module is used to acquire images within a set road area and detect obstacles to be identified in the images;

[0011] The obstacle recognition module is used to enter the obstacle recognition mode according to the current driving mode of the vehicle when the obstacle to be recognized meets the recognition conditions and the obstacle to be recognized cannot be recognized according to the existing obstacle model in the database, so as to obtain the recognition result of the obstacle to be recognized through interaction with the server.

[0012] Thirdly, embodiments of the present invention provide an electronic device, including:

[0013] At least one processor; and

[0014] A memory communicatively connected to the at least one processor; wherein,

[0015] The memory stores a computer program that can be executed by the at least one processor to enable the at least one processor to perform the obstacle recognition method as described in the first aspect.

[0016] Fourthly, embodiments of the present invention provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the obstacle recognition method as described in the first aspect.

[0017] The technical solution of this invention improves the obstacle recognition capability of the entire interactive system by interacting with the server to identify the obstacle when the obstacle meets the recognition conditions and the existing obstacle model in the database cannot identify the obstacle.

[0018] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a flowchart of an obstacle recognition method provided according to Embodiment 1 of the present invention;

[0021] Figure 2 This is a flowchart for determining whether an obstacle to be identified meets the identification conditions according to Embodiment 1 of the present invention;

[0022] Figure 3 This is a flowchart of an obstacle recognition method provided according to Embodiment 2 of the present invention;

[0023] Figure 4 This is a flowchart of controlling the relative speed between the vehicle and the obstacle to be identified according to the current driving mode of the vehicle, provided in Embodiment 2 of the present invention;

[0024] Figure 5 This is a flowchart of constructing an obstacle model to be identified based on appearance features and operational features, according to Embodiment 2 of the present invention;

[0025] Figure 6 This is a flowchart of naming the obstacle model to be identified according to Embodiment 2 of the present invention;

[0026] Figure 7 This is a flowchart of another obstacle recognition method provided according to Embodiment 2 of the present invention;

[0027] Figure 8 This is a schematic diagram of the interaction between multiple vehicles and the server according to Embodiment 2 of the present invention;

[0028] Figure 9 This is a schematic diagram of the structure of an obstacle recognition device according to Embodiment 3 of the present invention;

[0029] Figure 10 This is a schematic diagram of the structure of an electronic device that implements the obstacle recognition method of this invention. Detailed Implementation

[0030] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0031] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0032] It is understood that before using the technical solutions disclosed in the various embodiments of the present invention, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this disclosure in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained.

[0033] Example 1

[0034] Figure 1 This is a flowchart of an obstacle recognition method provided in Embodiment 1 of the present invention. This embodiment is applicable to situations involving obstacle recognition. The method can be executed by an obstacle recognition device, which can be implemented in software and / or hardware and integrated into an electronic device. Further, the electronic device includes, but is not limited to, computers, laptops, smartphones, servers, etc. Figure 1 As shown, the method includes:

[0035] S110. Acquire images within the designated road area and detect obstacles to be identified in the images.

[0036] Specifically, defining the road area refers to the area of ​​the road used for image acquisition. Obstacles to be identified refer to objects that may impede vehicle movement and need to be identified.

[0037] The method of setting the road range is not limited. For example, it can be set according to actual needs; or it can be preset in advance, such as the range within a preset distance in front of the vehicle. The preset distance can be a distance set according to actual needs.

[0038] The method of acquiring images within the designated road area is not limited here, as long as it can acquire images within the designated road area. For example, images within the designated road area can be acquired using an image acquisition device, which can be a camera or a combination of a sensor and a camera, etc. The sensor can be a distance sensor or an image sensor, etc.

[0039] The acquisition time for images within the set road area is not limited; for example, it can be continuous acquisition. Alternatively, images within the set road area can be acquired by setting a time interval, where the time interval can be a time interval set according to actual needs, such as 1 second.

[0040] The type of obstacle to be identified is not limited. It can be a moving object, such as a moving vehicle or a moving non-motorized vehicle; or a stationary object, such as a tree, road post or stone next to the road.

[0041] The method for detecting obstacles in an image is not limited, as long as it can detect the obstacles in the image. For example, an area threshold can be set to determine whether there are obstacles in the image within the acquired road area. When the area occupied by an object in the image exceeds the set area threshold, it indicates that there is an obstacle in the image. Similarly, a length threshold and / or a width threshold can be set to determine whether there are obstacles in the image within the acquired road area. When the length of an object in the image exceeds the length threshold and / or the width exceeds the width threshold, it indicates that there is an obstacle in the image.

[0042] S120. When the obstacle to be identified meets the identification conditions but cannot be identified based on the existing obstacle model in the database, the vehicle enters the obstacle identification mode according to the current driving mode of the vehicle, so as to obtain the identification result of the obstacle to be identified through interaction with the server.

[0043] Specifically, recognition conditions can refer to the conditions that enable obstacle recognition. A database can refer to a "repository" that organizes, stores, and manages data according to a data structure. An obstacle model can refer to a collection of information representing obstacles. An obstacle recognition pattern can refer to a pattern for recognizing unrecognizable obstacles. Recognition result can refer to the outcome of obstacle recognition.

[0044] The method of determining the recognition conditions is not limited. For example, the user can set the recognition conditions through the human-computer interaction device of the electronic device; or the recognition conditions can be saved in the electronic device and requested to be called by the electronic device when the obstacle needs to be recognized.

[0045] Optional, the identification criteria include:

[0046] The feature points of the obstacle to be identified appear in multiple consecutive frames of images, and the area occupied by the feature points increases frame by frame.

[0047] Feature points, in image processing, can refer to points where the grayscale value of an image changes drastically or points with significant curvature at the edges of an image (i.e., the intersection of two edges). A feature point consists of two parts: a keypoint and a descriptor. A keypoint refers to the location of the feature point in the image; some feature points also possess information such as orientation and size. A descriptor is typically a vector describing the information of the pixels surrounding the keypoint. Image feature points can reflect the essential features of an image and identify obstacles to be recognized within it.

[0048] The feature points of the obstacle to be identified appear in multiple consecutive frames of images. This can be understood as the presence of feature points that can characterize the obstacle to be identified in multiple consecutive frames of images acquired during continuous acquisition of images within a set road range, or when images within a set road range are acquired at set time intervals.

[0049] The area occupied by feature points increases frame by frame. This can be understood as the distance between the obstacle to be identified and the vehicle decreasing over time. In the image, this is represented by the area occupied by the feature points of the obstacle to be identified increasing frame by frame.

[0050] The feature points of the obstacle to be identified appear in multiple consecutive frames of images and the area occupied by the feature points increases frame by frame, indicating that the obstacle to be identified appears continuously near the vehicle and the distance between the obstacle to be identified and the vehicle is constantly decreasing, thus meeting the recognition conditions of the obstacle to be identified.

[0051] If the obstacle to be identified meets the identification conditions, it can be understood that the identification of the obstacle to be identified can begin. However, it is still necessary to combine the existing obstacle models in the database to determine whether the obstacle to be identified can be identified or not.

[0052] In one embodiment, Figure 2 The flowchart shown is a process for determining whether an obstacle to be identified meets the identification criteria. Figure 2 As shown, when the vehicle speed is greater than 3 km / h but less than 120 km / h, the system determines whether there are certain feature points in the image with an increasing area. If so, it determines whether the obstacle is clear; otherwise, the process ends. The clarity of the obstacle can be determined by the area or size it occupies in the image.

[0053] The obstacle models already present in the database can refer to obstacle models stored in advance, models built according to actual needs, or obstacle models downloaded from a webpage. The database can be a local database on an electronic device or in a vehicle, or a database built on a cloud server. The database may include, but is not limited to, obstacle models and corresponding avoidance strategies, where the avoidance strategy refers to the vehicle's strategy for avoiding obstacles.

[0054] The obstacle model may include, but is not limited to, obstacle shape information and obstacle type. The obstacle shape information may refer to information that can characterize the shape features of the obstacle, such as the length, width or appearance features of the obstacle. The obstacle type may include, but is not limited to, trucks, cars, special operation vehicles, trees, rocks or non-motorized vehicles.

[0055] There are no restrictions on the method of identifying the obstacle to be identified based on existing obstacle models in the database, as long as the obstacle can be identified using existing obstacle models in the database. For example, it can be determined whether the information of the obstacle to be identified can be matched with existing obstacle models in the database. If they match, it means that the obstacle to be identified can be identified based on existing obstacle models in the database; otherwise, the obstacle to be identified cannot be identified.

[0056] In one embodiment, identifying an obstacle to be identified based on existing obstacle models in a database can be achieved by determining the similarity between the information of the obstacle to be identified and the information of obstacles stored in the existing obstacle models in the database. Specifically, information about the obstacle to be identified can be extracted from images within a defined road area captured by an electronic device and compared with the information of obstacles stored in the obstacle models in the database to determine whether the obstacle to be identified can be identified. The obstacle information can refer to information that characterizes the obstacle, such as its length, width, or appearance. When the similarity between the information of the obstacle to be identified and the information of obstacles stored in the obstacle models exceeds a set threshold, it indicates that the obstacle to be identified can be identified based on the existing obstacle models in the database; otherwise, the obstacle to be identified cannot be identified. For example, if the appearance of the obstacle to be identified is a truck, and the similarity of one or more of the length, width, or height of the obstacle to be identified with the information of obstacles stored in the existing obstacle models in the database exceeds a set threshold, then the obstacle to be identified can be identified as a truck; if the appearance, length, width, or height of the obstacle to be identified are inconsistent with the information of obstacles stored in the existing obstacle models in the database, then the obstacle to be identified cannot be identified.

[0057] In one embodiment, identifying an obstacle to be identified based on an existing obstacle model in a database is achieved by dividing an image into regions and comparing similar features. Specifically, the existing obstacle model in the database may store images of multiple obstacles, each image of which is divided into multiple small regions. When detecting an obstacle to be identified in an image within a defined road area, the image of the obstacle to be identified is divided into multiple small regions, and then compared with the images of the multiple obstacles stored in the existing obstacle model in the database. If multiple small regions of the image of the obstacle to be identified have overlapping similar features with multiple small regions of the images of the obstacles stored in the obstacle model, it indicates that the obstacle to be identified can be identified; otherwise, the obstacle to be identified cannot be identified.

[0058] If the obstacle to be identified meets the identification conditions but cannot be identified based on the existing obstacle models in the database, it can be understood that the identification of the obstacle to be identified can begin, but the obstacle to be identified cannot be identified based on the existing obstacle models in the database.

[0059] The vehicle enters obstacle recognition mode based on its current driving mode to obtain the recognition results of obstacles through interaction with the server. Driving modes may include, but are not limited to, autonomous driving mode and manual driving mode. The method of obstacle recognition differs depending on the current driving mode. When entering obstacle recognition mode, obstacles can be modeled, their information saved, and uploaded to the server to obtain the recognition results.

[0060] The technical solution of this invention improves the obstacle recognition capability of the entire interactive system by detecting obstacles to be identified in images within a set road range, and when the obstacle to be identified meets the recognition conditions and the existing obstacle model in the database cannot identify the obstacle, interacting with the server to identify the obstacle.

[0061] Furthermore, obstacle recognition methods also include:

[0062] When the feature points of the obstacle to be identified meet the identification conditions, if there is an obstacle model in the database that matches the feature points, the obstacle model is called and the vehicle is controlled to avoid the obstacle to be identified according to the avoidance strategy corresponding to the obstacle model.

[0063] Specifically, the existence of obstacle models matching the feature points in the database indicates that the database contains information about the obstacle to be identified and has an avoidance strategy for the obstacle. The obstacle model corresponding to the obstacle to be identified can be called, and the vehicle can be controlled to avoid the obstacle according to the avoidance strategy corresponding to the obstacle model. This effectively improves the efficiency of obstacle identification and avoidance, and increases the safety factor of driving the vehicle.

[0064] Example 2

[0065] Figure 3 This is a flowchart of an obstacle recognition method provided in Embodiment 2 of the present invention. This embodiment is based on Embodiment 1 above. When the obstacle to be recognized meets the recognition conditions but cannot be recognized according to existing obstacle models in the database, the method enters an obstacle recognition mode based on the vehicle's current driving mode. This allows for the refinement of the obstacle recognition results through interaction with the server, such as... Figure 3 As shown, the method includes:

[0066] S110. Acquire images within the designated road area and detect obstacles to be identified in the images.

[0067] S121. When the obstacle to be identified meets the identification conditions but cannot be identified according to the existing obstacle model in the database, the relative speed between the vehicle and the obstacle to be identified is controlled to be within the speed range specified by the obstacle identification mode according to the current driving mode of the vehicle.

[0068] The method of controlling the relative speed between the vehicle and the obstacle to be identified based on the vehicle's current driving mode is not limited, as long as the relative speed between the vehicle and the obstacle can be controlled. For example, in autonomous driving mode, the relative speed between the vehicle and the obstacle can be controlled by combining speed sensors and / or distance sensors; or in manual driving mode, the instrument panel can prompt the driver to control the relative speed between the vehicle and the obstacle.

[0069] The speed range specified by the obstacle recognition mode can refer to the range of relative speeds between the vehicle and the obstacle to be recognized when the obstacle is being recognized. When the relative speed between the vehicle and the obstacle to be recognized is within the speed range specified by the obstacle recognition mode, the recognition of the obstacle can be maintained.

[0070] The method for specifying the speed range is not limited, as long as it ensures that obstacle recognition can be maintained within the specified speed range. For example, the speed range can be set according to actual needs; or, the speed range can be specified based on the ability to recognize obstacles in the images of the selected road area.

[0071] S122. While the vehicle is traveling at a relative speed, collect the appearance and operational characteristics of the obstacle to be identified.

[0072] The appearance features of an obstacle to be identified can refer to features that characterize the appearance of the obstacle. These appearance features may include, but are not limited to, the rear, side, and front features of the obstacle. Collecting the rear, side, and front features of an obstacle while the vehicle is traveling at a constant relative speed can improve its identification.

[0073] The operational characteristics of an obstacle to be identified can refer to features that characterize the working process of the obstacle. For example, when the obstacle to be identified is a special-purpose vehicle, different special-purpose vehicles have different uses, and their operational characteristics also differ. Special-purpose vehicles refer to vehicles whose external dimensions, weight, etc., exceed the design vehicle limits and are for special purposes. They are specially made or modified, equipped with fixed devices and equipment, and whose primary function is not to carry people or goods. Examples include ambulances, inspection vehicles, fire trucks, and water trucks.

[0074] There are no restrictions on the method used to collect the appearance and operational characteristics of the obstacle to be identified, as long as these characteristics can be collected. For example, the appearance and operational characteristics of the obstacle to be identified can be collected by capturing images with a camera.

[0075] In one embodiment, the obstacle to be identified is a water truck. While the vehicles are traveling at a relative speed, the appearance and operational characteristics of the water truck are collected. The appearance characteristics of the water truck can include features such as a water tank, front spray valve, side spray valve, rear spray valve, or other features that can identify the obstacle as a water truck. The operational characteristics of the water truck can include front spraying, rear spraying, or side spraying. Front spraying refers to water spraying from the front of the water truck, which can be used for washing streets, suppressing dust, and cooling; rear spraying refers to water spraying from the rear of the water truck, which can be used for watering engineering roads; side spraying refers to water spraying from the sides of the water truck, which can be used for watering flower beds, low trees, green belts, and lawns.

[0076] S123. Send the obstacle model to be identified, constructed based on appearance and operational characteristics, to the server and obtain the identification results of the obstacle to be identified.

[0077] The obstacle model to be identified can refer to a set of information representing the obstacle to be identified. The obstacle model to be identified may include, but is not limited to, the shape information and type of the obstacle.

[0078] The obstacle model to be identified, constructed based on appearance and operational features, can be understood as storing the appearance and operational features of the obstacle to be identified in the obstacle model. When it is necessary to identify an obstacle, a matching obstacle can be found in the obstacle model based on the appearance and operational features.

[0079] The identification results of the obstacle to be identified may include, but are not limited to, the corrected naming of the obstacle model and the avoidance strategy corresponding to the obstacle model.

[0080] Sending the obstacle model to the server and obtaining the recognition result can be understood as sending the obstacle model to the server, the server receiving the obstacle model and responding to it, and performing corresponding operations, such as naming the obstacle model, saving the avoidance strategy corresponding to the obstacle model, or correcting the name of the obstacle model.

[0081] Furthermore, based on the vehicle's current driving mode, the relative speed between the vehicle and the obstacle to be identified is controlled to be within the speed range specified by the obstacle recognition mode, including:

[0082] If the vehicle's current driving mode is automatic driving mode, then control the vehicle's drive torque so that the relative speed between the vehicle and the obstacle to be identified is within the speed range specified by the obstacle recognition mode.

[0083] If the vehicle's current driving mode is manual driving mode, a prompt message is generated to remind the driver to take braking measures until the relative speed between the vehicle and the obstacle to be identified is within the speed range specified by the obstacle recognition mode.

[0084] Driving torque can refer to the torque that propels the vehicle, and torque can refer to the force output from the crankshaft of the engine. The prompt information can refer to information that serves to remind the driver; the form of the prompt information is not limited, such as displaying text on the dashboard or issuing an audible prompt. The method of generating the prompt information is not limited; for example, it can be generated by electronic devices detecting the current driving mode of the vehicle.

[0085] When the vehicle's current driving mode is autonomous driving mode, controlling the vehicle's drive torque can change the vehicle's speed, so that the relative speed between the vehicle and the obstacle to be identified is within the speed range specified by the obstacle identification mode. This can maintain the identification of the obstacle and facilitate the collection of the obstacle's appearance and operational characteristics.

[0086] When the vehicle's current driving mode is manual driving mode, a prompt message is generated to remind the driver to take braking measures. By changing the vehicle's speed, the relative speed between the vehicle and the obstacle to be identified is kept within the speed range specified by the obstacle identification mode, thereby maintaining the identification of the obstacle.

[0087] In one embodiment, Figure 4 The diagram shows a flowchart of controlling the relative speed between the vehicle and the obstacle to be identified based on the vehicle's current driving mode. Figure 4As shown, when the obstacle to be identified meets the identification conditions, it is determined whether the obstacle can be identified based on the existing obstacle model in the database. If yes, the obstacle model is invoked and the vehicle is controlled to avoid the obstacle according to the avoidance strategy corresponding to the obstacle model, and then the process ends. If no, the instrument panel indicates that the obstacle cannot be identified, and it is determined whether the vehicle's driving mode is in automatic driving mode. If yes, the vehicle's drive torque is controlled to make the relative speed between the vehicle and the obstacle within the speed range specified by the obstacle identification mode, and then the appearance and operational characteristics of the obstacle are collected (photos are taken and saved from the back / side / front), and the operation ends. If no, it is determined whether the relative speed between the vehicle and the obstacle is within the speed range specified by the obstacle identification mode. If yes, the appearance and operational characteristics of the obstacle are collected (photos are taken and saved from the back / side / front), and the operation ends. If no, the operation ends directly.

[0088] Furthermore, the appearance features include rear features, side features, and front features;

[0089] Collect the appearance features of the obstacle to be identified, including:

[0090] The rear features of the obstacle to be identified are determined based on images taken before the vehicle passes the obstacle; the side features of the obstacle to be identified are determined based on images taken from the moment the vehicle begins to pass the obstacle until the passing of the obstacle is completed; and the front features of the obstacle to be identified are determined based on images taken after the vehicle passes the obstacle.

[0091] Specifically, before the vehicle overtakes the obstacle, it is positioned behind the obstacle, allowing the rear features of the obstacle to be determined by capturing images. From the moment the vehicle begins to overtake the obstacle until it completes the overtaking maneuver, the vehicle's position relative to the obstacle changes from its rear to its front, allowing the side features of the obstacle to be determined by capturing images. After overtaking the obstacle, the vehicle is positioned in front of the obstacle, allowing the front features of the obstacle to be determined by capturing images. By capturing the entire process of the vehicle overtaking the obstacle, the appearance features of the obstacle can be completely captured, facilitating subsequent modeling of the obstacle.

[0092] Furthermore, the obstacle model to be identified, constructed based on appearance and operational characteristics, is sent to the server, and the identification results of the obstacle to be identified are obtained, including:

[0093] Construct a model of the obstacle to be identified based on its appearance and operational characteristics;

[0094] Search the database for target obstacle models; the similarity between the target obstacle model and the obstacle model to be identified is greater than a set threshold.

[0095] Name the obstacle model to be identified based on the target obstacle model;

[0096] The obstacle model to be identified is sent to the server, and the identification results of the obstacle to be identified are obtained.

[0097] In one embodiment, Figure 5 The diagram shows a flowchart for constructing a model of the obstacle to be identified based on its appearance and operational characteristics. Figure 5 As shown, when entering obstacle recognition mode, the obstacle to be identified is modeled based on images captured during the acquisition of its appearance and operational characteristics. The rear, side, and front features of the obstacle are modeled separately based on rear, side, and front photos. The operational characteristics of the obstacle are modeled based on photos of human factors and construction factors. Human factor photos may include human intervention, while construction factor photos may show the obstacle being operated. For example, if the obstacle is a water truck, the human factor photos may show a person controlling the truck, and the construction factor photos may show the truck washing the street with its front sprayer. Once the appearance and operational characteristics of the obstacle are modeled, the obstacle model is stored, along with the route and speed information for the entire process of avoiding the obstacle, and the operation ends. The route information may include the length of the route taken by the vehicle to avoid the obstacle, and the speed information may refer to the vehicle's speed or acceleration during the obstacle avoidance.

[0098] The process involves searching for target obstacle models in a database. A target obstacle model is defined as one whose similarity to the obstacle model to be identified exceeds a set threshold. Specifically, the database stores multiple obstacle models. A target obstacle model is any obstacle model in the database whose similarity to the obstacle model to be identified exceeds the set threshold. The method for determining similarity is not limited; it can be based on the obstacle's appearance or operational characteristics stored in the obstacle model. The set threshold is also not limited; it can be set according to actual needs. The method for searching for target obstacle models in the database is not limited. For example, it could involve iterating through each obstacle model in the database, comparing its similarity to the obstacle model to be identified, and then considering the obstacle model with a similarity exceeding the set threshold as the target obstacle model and uploading it to the server.

[0099] There are no restrictions on how the obstacle model to be identified is named based on the target obstacle model, as long as it can be named. For example, the server can automatically name the obstacle model to be identified based on one or more of the appearance features or operational features that are highly similar to the target obstacle model; or the server can control a voice assistant to request the driver to name the obstacle model to be identified, making the naming of the obstacle model to be identified more personalized.

[0100] By constructing a model of the obstacle to be identified and naming the model based on the target obstacle model, the model is sent to the server to obtain the identification results of the obstacle, making it more convenient to identify the obstacle.

[0101] Furthermore, the identification results include the corrected name of the obstacle model to be identified and the avoidance strategy corresponding to the obstacle model to be identified;

[0102] After obtaining the recognition results of the obstacle to be identified, the method also includes:

[0103] The names of the obstacle models to be identified are updated based on the identification results, and the obstacle models to be identified and their corresponding avoidance strategies are added to the database.

[0104] The corrected naming of the obstacle model to be identified can refer to the server renaming the obstacle model based on its learning process. The avoidance strategy corresponding to the obstacle model can refer to the vehicle's strategy for avoiding the obstacle.

[0105] After obtaining the recognition results of the obstacle to be identified, the name of the obstacle model to be identified is updated according to the recognition results, and the obstacle model to be identified and the corresponding avoidance strategy are added to the database. When the vehicle encounters the obstacle to be identified again, the obstacle model to be identified and the corresponding avoidance strategy can be retrieved from the database to identify and avoid the obstacle, thereby improving the safety performance of vehicle driving.

[0106] Optionally, if no model with a similarity greater than a set threshold is found in the database, the obstacle model to be identified can be named a general model, and the obstacle model to be identified and the corresponding avoidance strategy can be added to the database.

[0107] In one embodiment, Figure 6 The diagram shows a flowchart for naming the obstacle models to be identified, such as... Figure 6As shown, the system first analyzes camera and vehicle speed data. If the vehicle is currently operating at high speed, it calls the high-speed obstacle model set. Otherwise, it determines whether the vehicle is operating in a suburban area. If so, it calls the suburban obstacle model set; otherwise, it calls the urban obstacle model set. Next, it checks if a model with high similarity to the obstacle to be identified can be found in the high-speed, suburban, and urban obstacle model sets. If so, the model with high similarity is used as the target obstacle model, and the obstacle to be identified is named accordingly. If not, the obstacle to be identified is an unknown obstacle, and unknown obstacles are named sequentially. Then, the system automatically generates a name for the obstacle to be identified or requests a name from the driver via voice. The system saves the obstacle model and speed information and uploads it to the server. Finally, the system provides the name via voice feedback from the driver or the processor, and the processor provides the obstacle avoidance route information and control logic. Finally, based on the learning of the obstacle model to be identified, the server corrects the naming of the obstacle model to be identified and adds the obstacle model to be identified and the corresponding avoidance strategy to the database.

[0108] In one embodiment, Figure 7 A flowchart of yet another obstacle recognition method is provided, such as Figure 7 As shown, when an obstacle to be identified is encountered ahead and the identification conditions are met, it is determined whether the obstacle to be identified can be identified based on the existing obstacle models in the database. If so, the corresponding obstacle model is called, and the speed and route information for avoiding the obstacle to be identified stored in the database are obtained. The control logic and avoidance strategy are then obtained, and the vehicle is controlled to avoid the obstacle to be identified according to the avoidance strategy corresponding to the obstacle model. If not, the obstacle to be identified is modeled, and a target obstacle model or general model with high similarity to the obstacle model to be identified is displayed. The speed and route information for avoiding the obstacle to be identified are then learned, the control logic and avoidance strategy are analyzed, and finally the speed and route information, as well as the control logic and avoidance strategy, are uploaded to the database.

[0109] In one embodiment, Figure 8 A diagram illustrating the interaction between multiple vehicles and the server is provided, such as... Figure 8 As shown, multiple vehicles 1 to vehicle n are connected to the server and can interact with the server. The number of vehicles n can be set according to actual needs.

[0110] The server stores a database containing obstacle models and corresponding avoidance strategies. When a vehicle encounters an obstacle to be identified and meets the identification conditions, the vehicle requests the server to query the existing obstacle models in the database. Based on the existing obstacle models in the database, the vehicle determines whether the obstacle to be identified can be identified. If it can be identified, the vehicle calls the corresponding obstacle model and the corresponding avoidance strategy to identify and avoid the obstacle.

[0111] If the obstacle cannot be identified, a learning model is created to generate an obstacle model, which is then uploaded to the database. Simultaneously, the vehicle's speed and route information when avoiding the obstacle are saved and uploaded to the server. The server analyzes the control logic and avoidance strategy, and uploads the speed and route information to the database. The database stored on the server will then contain the obstacle model and its corresponding avoidance strategy.

[0112] The obstacle models and corresponding avoidance strategies stored in the database on the server can be queried by any vehicle interacting with the server, allowing them to avoid obstacles using the strategies. When one vehicle interacts with the server to add an obstacle model and its corresponding avoidance strategy to the database, other vehicles can also obtain the same obstacle model and its avoidance strategy by interacting with the server. Multiple vehicles interacting with the server continuously update the obstacle models in the database, and the ability for multiple vehicles to share these models improves obstacle identification and enhances driving safety.

[0113] The technical solution of this invention has different ways of collecting the appearance and operation features of the obstacle to be identified in different vehicle driving modes, so that the vehicle and the obstacle to be identified maintain a certain relative speed. The obstacle to be identified is modeled and named according to the collected appearance and operation features, so as to facilitate the subsequent retrieval of the obstacle model in the database.

[0114] Example 3

[0115] Figure 9 This is a schematic diagram of an obstacle recognition device provided in Embodiment 3 of the present invention. This embodiment is applicable to situations requiring obstacle recognition. Figure 9 As shown, the specific structure of the device includes:

[0116] The obstacle detection module 21 is used to acquire images within a set road area and detect obstacles to be identified in the images;

[0117] The obstacle recognition module 22 is used to enter the obstacle recognition mode according to the current driving mode of the vehicle when the obstacle to be recognized meets the recognition conditions and the obstacle to be recognized cannot be recognized according to the existing obstacle model in the database, so as to obtain the recognition result of the obstacle to be recognized through interaction with the server.

[0118] The obstacle recognition device provided in this embodiment first acquires images within a set road area through the obstacle detection module 21 and detects obstacles to be recognized in the images; then, through the obstacle recognition module 22, when the obstacle to be recognized meets the recognition conditions and the obstacle to be recognized cannot be recognized according to the existing obstacle model in the database, the obstacle recognition mode is entered according to the current driving mode of the vehicle, so as to obtain the recognition result of the obstacle to be recognized through interaction with the server.

[0119] Furthermore, the recognition conditions in obstacle recognition module 22 include:

[0120] The feature points of the obstacle to be identified appear in multiple consecutive frames of images, and the area occupied by the feature points increases frame by frame.

[0121] Furthermore, the obstacle recognition module 22 is specifically used for:

[0122] The vehicle's relative speed to the obstacle to be identified is controlled within the speed range specified by the obstacle recognition mode based on the vehicle's current driving mode.

[0123] While the vehicle is moving at a relative speed, the appearance and operational characteristics of the obstacle to be identified are collected.

[0124] The obstacle model, constructed based on appearance and operational characteristics, is sent to the server, and the recognition results of the obstacle are obtained.

[0125] Furthermore, the obstacle recognition module 22 is specifically used for:

[0126] If the vehicle's current driving mode is automatic driving mode, then control the vehicle's drive torque so that the relative speed between the vehicle and the obstacle to be identified is within the speed range specified by the obstacle recognition mode.

[0127] If the vehicle's current driving mode is manual driving mode, a prompt message is generated to remind the driver to take braking measures until the relative speed between the vehicle and the obstacle to be identified is within the speed range specified by the obstacle recognition mode.

[0128] Furthermore, the obstacle recognition module 22 is specifically used for:

[0129] The rear features of the obstacle to be identified are determined based on images taken before the vehicle passes the obstacle; the side features of the obstacle to be identified are determined based on images taken from the moment the vehicle begins to pass the obstacle until the passing is completed; and the front features of the obstacle to be identified are determined based on images taken after the vehicle passes the obstacle. The appearance features include rear, side, and front features.

[0130] Furthermore, the obstacle recognition module 22 is specifically used for:

[0131] Construct a model of the obstacle to be identified based on its appearance and operational characteristics;

[0132] Search the database for target obstacle models; the similarity between the target obstacle model and the obstacle model to be identified is greater than a set threshold.

[0133] Name the obstacle model to be identified based on the target obstacle model;

[0134] The obstacle model to be identified is sent to the server, and the identification results of the obstacle to be identified are obtained.

[0135] Furthermore, after obtaining the recognition result of the obstacle to be identified, the obstacle recognition module 22 is specifically used for:

[0136] The names of the obstacle models to be identified are updated based on the identification results, and the obstacle models and their corresponding avoidance strategies are added to the database. The identification results include the corrected names of the obstacle models and the corresponding avoidance strategies.

[0137] Furthermore, the device also includes:

[0138] The obstacle model calling module 23 is used to call the obstacle model and control the vehicle to avoid the obstacle to be identified according to the avoidance strategy corresponding to the obstacle model when the feature points of the obstacle to be identified meet the recognition conditions. If there is an obstacle model in the database that matches the feature points, the obstacle model is called.

[0139] The obstacle recognition device provided in the embodiments of the present invention can execute the obstacle recognition method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the method execution.

[0140] Example 4

[0141] Figure 10A schematic diagram of an electronic device 10 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0142] like Figure 10 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 may also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0143] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0144] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as obstacle recognition methods.

[0145] In some embodiments, the obstacle recognition method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or mounted on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the obstacle recognition method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the obstacle recognition method by any other suitable means (e.g., by means of firmware).

[0146] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0147] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0148] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0149] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0150] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0151] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0152] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0153] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. An obstacle recognition method, characterized in that, include: Acquire images within a defined road area and detect obstacles to be identified in the images; When the obstacle to be identified meets the identification conditions but cannot be identified according to the existing obstacle model in the database, the vehicle enters the obstacle identification mode according to the current driving mode of the vehicle, so as to obtain the identification result of the obstacle to be identified through interaction with the server. Among them, entering obstacle recognition mode based on the vehicle's current driving mode includes: The relative speed between the vehicle and the obstacle to be identified is controlled within the speed range specified by the obstacle identification mode according to the current driving mode of the vehicle. While the vehicle is traveling at the relative speed, the appearance features and operational features of the obstacle to be identified are collected; wherein, the appearance features include the rear, side, and front features of the obstacle to be identified; the operational features include features that characterize the working process of the obstacle to be identified; The obstacle model to be identified, constructed based on the appearance features and the operation features, is sent to the server, and the identification result of the obstacle to be identified is obtained.

2. The method according to claim 1, characterized in that, The identification conditions include: The feature points of the obstacle to be identified appear in multiple consecutive frames of images, and the area occupied by the feature points increases frame by frame.

3. The method according to claim 1, characterized in that, Controlling the relative speed between the vehicle and the obstacle to be identified within the speed range specified by the obstacle recognition mode according to the vehicle's current driving mode includes: If the current driving mode of the vehicle is autonomous driving mode, then control the driving torque of the vehicle so that the relative speed between the vehicle and the obstacle to be identified is within the speed range specified by the obstacle identification mode. If the vehicle's current driving mode is manual driving mode, a prompt message is generated to remind the driver to take braking measures until the relative speed between the vehicle and the obstacle to be identified is within the speed range specified by the obstacle identification mode.

4. The method according to claim 1, characterized in that, Collect the appearance features of the obstacle to be identified, including: The rear features of the obstacle to be identified are determined based on an image taken before the vehicle passes the obstacle to be identified. The side features of the obstacle to be identified are determined based on images taken from the moment the vehicle begins to pass the obstacle until the passage of the obstacle is completed; The frontal features of the obstacle to be identified are determined based on an image taken after the vehicle has passed the obstacle.

5. The method according to claim 1, characterized in that, The obstacle model to be identified, constructed based on the appearance features and operational features, is sent to the server, and the identification result of the obstacle to be identified is obtained, including: The model of the obstacle to be identified is constructed based on the appearance features and the operational features; The target obstacle model is searched in the database, and the similarity between the target obstacle model and the obstacle model to be identified is greater than a set threshold. Name the obstacle model to be identified based on the target obstacle model; The obstacle model to be identified is sent to the server, and the identification result of the obstacle to be identified is obtained.

6. The method according to claim 5, characterized in that, The identification results include the corrected name of the obstacle model to be identified and the avoidance strategy corresponding to the obstacle model to be identified; After obtaining the identification result of the obstacle to be identified, the method further includes: The name of the obstacle model to be identified is updated according to the identification result, and the obstacle model to be identified and the avoidance strategy corresponding to the obstacle model to be identified are added to the database.

7. The method according to claim 1, characterized in that, Also includes: When the feature points of the obstacle to be identified meet the identification conditions, if there is an obstacle model in the database that matches the feature points, then the obstacle model is invoked and the vehicle is controlled to avoid the obstacle to be identified according to the avoidance strategy corresponding to the obstacle model.

8. An obstacle recognition device, characterized in that, include: The obstacle detection module is used to acquire images within a set road area and detect obstacles to be identified in the images; The obstacle recognition module is used to enter the obstacle recognition mode according to the current driving mode of the vehicle when the obstacle to be recognized meets the recognition conditions and the obstacle to be recognized cannot be recognized according to the existing obstacle model in the database, so as to obtain the recognition result of the obstacle to be recognized through interaction with the server. Specifically, the obstacle recognition module is used for: The relative speed between the vehicle and the obstacle to be identified is controlled within the speed range specified by the obstacle identification mode according to the current driving mode of the vehicle. While the vehicle is traveling at the relative speed, the appearance features and operational features of the obstacle to be identified are collected; wherein, the appearance features include the rear, side, and front features of the obstacle to be identified; the operational features include features that characterize the working process of the obstacle to be identified; The obstacle model to be identified, constructed based on the appearance features and the operation features, is sent to the server, and the identification result of the obstacle to be identified is obtained.

9. An electronic device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor to enable the at least one processor to perform the obstacle recognition method as described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the obstacle recognition method as described in any one of claims 1-7.

Images