Information generation method and device, equipment, storage medium and vehicle

Abstract

The invention relates to an information generation method and device, equipment, a storage medium and a vehicle. The method comprises the following steps: inputting an environment image acquired by acquisition equipment into a bird's-eye view model to obtain three-dimensional attribute information of obstacle vehicles around the acquisition equipment; wherein the three-dimensional attribute information comprises a vehicle distance; screening the environment image according to the vehicle distance to obtain a target image corresponding to the obstacle vehicle; and performing vehicle safety identification on the target image to obtain safety attribute information of the obstacle vehicle. According to the embodiment of the invention, the target image corresponding to the obstacle vehicle is further determined based on the vehicle distance in the three-dimensional attribute information, and vehicle safety identification is performed on the target image, so that the safety attribute information of the obstacle vehicle detected through the aerial view model is obtained; the information obtained by the aerial view model is supplemented in the safety dimension, and the driving safety of the vehicle is improved.

Description

Technical Field

[0001] This disclosure relates to the field of vehicle technology, and more particularly to an information generation method, apparatus, device, storage medium, and vehicle. Background Technology

[0002] With the development of vehicle technology, the Bird's Eye View (BEV) function is becoming increasingly popular. The BEV function can provide a view of the scene from above the vehicle, and can be used to understand the situation of obstacles around the vehicle.

[0003] In related technologies, bird's-eye view functionality can identify the size and type of obstacles around a vehicle. However, this bird's-eye view functionality offers limited improvement to current vehicle driving safety, and how to enhance vehicle driving safety based on bird's-eye view functionality is an urgent problem to be solved. Summary of the Invention

[0004] To address the aforementioned technical problems, this disclosure provides an information generation method, apparatus, device, storage medium, and vehicle.

[0005] In a first aspect, this disclosure provides an information generation method, the method comprising:

[0006] The environmental images collected by the acquisition device are input into the bird's-eye view model to obtain the three-dimensional attribute information of obstacle vehicles around the acquisition device; wherein, the three-dimensional attribute information includes the vehicle distance;

[0007] The environmental images are filtered based on the vehicle distance to obtain the target image corresponding to the obstacle vehicle;

[0008] Vehicle safety identification is performed on the target image to obtain the safety attribute information of the obstacle vehicle.

[0009] Secondly, this disclosure provides an information generation apparatus, which includes:

[0010] The input module is used to input environmental images collected by the acquisition device into a bird's-eye view model to obtain three-dimensional attribute information of obstacle vehicles around the acquisition device; wherein, the three-dimensional attribute information includes vehicle distance;

[0011] The filtering module is used to filter the environmental image based on the vehicle distance to obtain the target image corresponding to the obstacle vehicle;

[0012] The safety module is used to perform vehicle safety identification on the target image and obtain the safety attribute information of the obstacle vehicle.

[0013] Thirdly, embodiments of this disclosure also provide an electronic device, including:

[0014] processor;

[0015] Memory, used to store executable instructions;

[0016] The processor is used to read executable instructions from memory and execute the executable instructions to implement the information generation method of the first aspect mentioned above.

[0017] Fourthly, embodiments of this disclosure also provide a computer-readable storage medium storing a computer program that, when executed by a processor, causes the processor to implement the information generation method described in the first aspect.

[0018] The technical solution provided in this disclosure has the following advantages compared with the prior art: An information generation method, apparatus, device, and storage medium according to this disclosure includes: inputting an environmental image collected by a data acquisition device into a bird's-eye view model to obtain three-dimensional attribute information of obstacle vehicles around the data acquisition device; wherein the three-dimensional attribute information includes vehicle distance; filtering the environmental image according to the vehicle distance to obtain a target image corresponding to the obstacle vehicle; and performing vehicle safety identification on the target image to obtain safety attribute information of the obstacle vehicle. By adopting the above technical solution, after processing the environmental image through the bird's-eye view model to obtain the three-dimensional attribute information of obstacle vehicles around the vehicle, the target image corresponding to the obstacle vehicle is further determined based on the vehicle distance in the three-dimensional attribute information. By performing vehicle safety identification on the target image, the safety attribute information of the obstacle vehicle detected by the bird's-eye view model is obtained, supplementing the information obtained by the bird's-eye view model in the safety dimension and improving vehicle driving safety. Attached Figure Description

[0019] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0020] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 A flowchart illustrating an information generation method provided in an embodiment of this disclosure;

[0022] Figure 2 A flowchart illustrating another information generation method provided in this embodiment of the disclosure;

[0023] Figure 3 This is a schematic diagram illustrating an embodiment of obtaining an image to be processed according to this disclosure.

[0024] Figure 4 A schematic diagram of a security identification model provided in an embodiment of this disclosure;

[0025] Figure 5 This is a schematic diagram illustrating the determination of vehicle attribute information provided in an embodiment of this disclosure;

[0026] Figure 6 This is a schematic diagram of the structure of an information generation device provided in an embodiment of the present disclosure;

[0027] Figure 7 This is a schematic diagram of the hardware circuit structure of an information generation device provided in an embodiment of this disclosure. Detailed Implementation

[0028] To better understand the above-mentioned objectives, features, and advantages of this disclosure, the solutions disclosed herein will be further described below. It should be noted that, unless otherwise specified, the embodiments and features described herein can be combined with each other.

[0029] Numerous specific details are set forth in the following description in order to provide a full understanding of this disclosure, but this disclosure may also be implemented in other ways different from those described herein; obviously, the embodiments in the specification are only some, and not all, of the embodiments of this disclosure.

[0030] To address the problems in the related technologies, embodiments of this disclosure provide an information generation method, apparatus, device, storage medium, and vehicle.

[0031] Figure 1 This is a flowchart illustrating an information generation method provided in an embodiment of the present disclosure. The method can be executed by an information generation device, which can be implemented using software and / or hardware, and is generally integrated into an electronic device. This information generation device and / or electronic device can be configured in a vehicle. Figure 1 As shown, the method includes:

[0032] Step 101: Input the environmental images collected by the acquisition device into the bird's-eye view model to obtain the three-dimensional attribute information of the obstacle vehicles around the acquisition device; wherein, the three-dimensional attribute information includes the vehicle distance.

[0033] The data acquisition device can be an environmental camera configured on the target vehicle, or it can be a camera independently set up separately from the target vehicle and equipped with a bird's-eye view generation function. This embodiment does not limit the number of data acquisition devices; there can be multiple devices. If there are multiple data acquisition devices, they can be considered as a whole, generating safety attribute information of the obstacle vehicle corresponding to this whole set of data acquisition devices.

[0034] The target vehicle can be a vehicle equipped with a bird's-eye view function. This embodiment does not limit the power supply type of the target vehicle; for example, the target vehicle can be powered by pure electric power or hybrid power. This embodiment also does not limit the type of the target vehicle; for example, the target vehicle can be a sport utility vehicle (SUV) or a sedan. The environmental camera can be an image sensor that acquires images of the external environment of the target vehicle. The environmental camera can be a camera facing outwards from the target vehicle. This environmental camera can be the camera used to achieve the bird's-eye view function of the target vehicle.

[0035] The environmental image can be an image captured by the acquisition device. This environmental image can be used to record the environment in which the acquisition device is located. If the acquisition device is an environmental camera configured for the target vehicle, then the environment recorded in the environmental image is the environment in which the target vehicle is located. The bird's-eye view model can be the processing model that realizes the conversion from two-dimensional image to three-dimensional model in the bird's-eye view function. Through the bird's-eye view model, objects located around the acquisition device are identified. If the acquisition device is an environmental camera configured for the target vehicle, then the objects identified by the bird's-eye view model are objects around the target vehicle. These objects can be regarded as obstacles, and obstacle vehicles can be obstacles whose object type is vehicles. This embodiment does not limit the number, location, etc. of these obstacle vehicles.

[0036] Three-dimensional attribute information can be used to characterize the obstacle vehicle's features in three-dimensional space, determined based on a three-dimensional bird's-eye view. Vehicle distance can be the three-dimensional spatial distance between the obstacle vehicle and the overall acquisition device within the bird's-eye view. If the acquisition device is an environmental camera configured for the target vehicle, the vehicle distance can be the three-dimensional spatial distance between the obstacle vehicle and the target vehicle within the bird's-eye view. This vehicle distance can be a straight-line distance, or it can include both lateral and longitudinal distances.

[0037] In this embodiment, the acquisition device acquires images in real time, generating corresponding environmental images. The information generation device acquires these environmental images and inputs them into a bird's-eye view model. The bird's-eye view model generates a bird's-eye view based on these images. In the bird's-eye view, obstacle vehicles around the acquisition device are identified, and their locations are determined using a three-dimensional bounding box. The three-dimensional bounding box can be cuboid in shape, and it can have up to eight vertex coordinates. Based on these vertex coordinates, the information generation device can determine the position of the obstacle vehicle in the bird's-eye view and calculate the distance between the obstacle vehicle and the acquisition device.

[0038] In some embodiments of this disclosure, the three-dimensional attribute type includes vehicle category. The information generation method further includes: removing obstacle vehicles whose vehicle category is outside a preset category range, and removing obstacle vehicles whose vehicle distance is outside a preset distance range. The preset category range can be a comprehensive range of category filtering ranges corresponding to each safety attribute information, and the preset distance range can be a comprehensive range of distance filtering ranges corresponding to each safety attribute information. This comprehensive range can be the union and intersection of the ranges. Therefore, before performing vehicle safety recognition on the target image, obstacle vehicles that do not meet the vehicle category and distance requirements are removed, avoiding a waste of computational power.

[0039] Step 102: Filter the environmental images according to the vehicle distance to obtain the target image corresponding to the obstacle vehicle.

[0040] The target image can be an environmental image that records vehicles with obstacles and is ultimately used for vehicle safety identification.

[0041] In this embodiment of the disclosure, the information generation device can filter multiple environmental images collected simultaneously based on vehicle distance to obtain the target image corresponding to the obstacle vehicle.

[0042] In some embodiments of this disclosure, environmental images are filtered based on vehicle distance to obtain target images corresponding to obstacle vehicles, including:

[0043] The environmental image of the vehicle with the obstacle is identified as a candidate image. If there are multiple candidate images and the distance between the vehicles is within the distance threshold, the largest detection box in the multiple detection boxes corresponding to the multiple candidate images is identified as the target detection box, and the candidate image containing the target detection box is identified as the target image.

[0044] The candidate image can be an environmental image recording vehicles with obstacles. The distance threshold range can be a pre-set threshold range used for the discrimination and filtering strategy. This embodiment does not limit the distance threshold range; for example, the distance threshold range can be 0-20 meters. The detection box can be a bounding box containing vehicles with obstacles. The size can include the side length or area.

[0045] In this embodiment, obstacle vehicle identification is achieved in the bird's-eye view. The information generation device can convert the three-dimensional bounding box of the obstacle vehicle in the bird's-eye view from three-dimensional to two-dimensional, and then convert the three-dimensional bounding box into a two-dimensional detection box. The environmental image containing the detection box is the environmental image recording the obstacle vehicle, and thus the environmental image containing the detection box is determined as a candidate image. It is determined whether there are multiple candidate images. If not, one candidate image is determined as the target image.

[0046] If there are multiple candidate images, it is determined whether the vehicle distance is within a distance threshold range. If it is, it indicates that the obstacle vehicle is relatively close to the acquisition device. In this case, the larger the size of the detection box, the larger the proportion of the obstacle vehicle in the candidate image, and the more obvious the relevant information of the obstacle vehicle recorded in the candidate image. Therefore, the information generation device can acquire multiple detection boxes corresponding to multiple candidate images, and determine the largest detection box among these multiple detection boxes as the target detection box, and determine the candidate image containing the target detection box as the target image. Thus, when the obstacle vehicle and the acquisition device are relatively close, the image with the largest proportion of the obstacle vehicle in the image is selected as the target image, making the features of the obstacle vehicle clearly displayed in the image, improving the accuracy of the safety attribute information extracted subsequently based on the target image.

[0047] In some embodiments of this disclosure, the information generation method further includes: if there are multiple candidate images and the vehicle distance is not within the distance threshold range, then the candidate images are filtered according to the camera orientation priority corresponding to the vehicle distance to obtain the target image; wherein, the forward-looking orientation takes precedence over the side-forward-looking orientation in the camera orientation priority.

[0048] The camera orientation priority can be a pre-set priority of the camera lens orientation. The forward-looking orientation can be the orientation facing the front of the target vehicle, and the side-forward-looking orientation can be the orientation facing the side of the target vehicle. The side-forward-looking orientation can include the left front orientation and / or the right front orientation. Optionally, the camera orientation priority can be arranged from high to low as follows: forward-looking orientation, left front orientation, and right front orientation.

[0049] In this embodiment, if there are multiple candidate images, it is determined whether the vehicle distance is within a distance threshold range. If not, it indicates that the distance between the obstacle vehicle and the target vehicle is relatively far. In this case, the forward-facing acquisition device can collect richer information about the obstacle vehicle. Therefore, the information generation device can acquire the camera lens orientation corresponding to multiple candidate images and filter the candidate images according to the camera orientation priority, determining the candidate image with the highest camera lens orientation priority as the target image. Thus, when the obstacle vehicle and the target vehicle are far apart, the image acquired by the acquisition device with the forward-facing camera lens is preferentially selected as the target image. This target image records more content related to the obstacle vehicle, improving the accuracy of subsequent extraction of safety attribute information.

[0050] In the above scheme, corresponding target image selection strategies are set according to different distance conditions, which improves the accuracy of subsequent security attribute information determined based on the target image.

[0051] Step 103: Perform vehicle safety identification on the target image to obtain the safety attribute information of the obstructing vehicle.

[0052] In this embodiment of the disclosure, the information generation device can use a neural network to perform vehicle safety identification on the obstacle vehicle in the target image and obtain the safety attribute information of the obstacle vehicle.

[0053] Figure 2 A flowchart illustrating another information generation method provided in this disclosure embodiment is shown below. Figure 2 As shown, in some embodiments of this disclosure, vehicle safety identification is performed on the target image to obtain safety attribute information of the obstructing vehicle, including:

[0054] Step 201: Extract the portion of the target image located within the target detection box to obtain the image to be processed.

[0055] The image to be processed can be an image for vehicle safety identification, and the image to be processed can be a portion of the target image located within the target detection box.

[0056] In this embodiment, the information generation device can capture a portion of the target image located within the target detection box to obtain the image to be processed.

[0057] In some embodiments of this disclosure, before cropping the portion of the target image located within the target detection box, the method further includes:

[0058] The target detection box is expanded to obtain an extended detection box, and the target detection box is then updated to the extended detection box. The extended detection box can be obtained by expanding the boundary line of the target detection box.

[0059] In this embodiment, the information generation device can expand the boundary line of the target detection box by a preset width and a preset length to obtain an expanded detection box, and use this expanded detection box as a new target detection box to replace the original target detection box. Therefore, the updated target detection box has a larger area than the original target detection box, enabling it to capture more image content. This allows the captured image to carry more information while avoiding information loss caused by an excessively small detection box.

[0060] Figure 3 This is a schematic diagram illustrating an embodiment of obtaining an image to be processed, as shown in the present disclosure. Figure 3 As shown, the information generation device can initially screen obstacle vehicles based on the vehicle type and distance recorded in the bird's-eye view, eliminating obstacle vehicles whose vehicle category and distance are outside the preset range. Furthermore, it performs a 2D-to-3D conversion on the obstacle vehicles identified in the bird's-eye view based on camera intrinsic parameters, and combines this with the environmental image to obtain detection boxes in each candidate image, determining the target image and its detection box within the candidate image. Further, the target detection box is expanded to obtain an updated target detection box, and the image to be processed within the target detection box undergoes standardization processing to meet the input requirements of the safety recognition model, resulting in a standardized image to be processed.

[0061] Step 202: Input the image to be processed into the trained safety recognition model to obtain safety attribute information; wherein, the safety attribute information includes at least two of the following: vehicle headlight attribute information, vehicle door attribute information, and line crossing attribute information.

[0062] The safety recognition model can be a multi-task neural network model capable of identifying vehicles with obstacles. The tasks performed by this safety attribute model can include determining one or more of the following: determining vehicle light attribute information, determining vehicle door attribute information, and determining lane marking attribute information. This safety recognition model can be trained based on labeled images; this embodiment does not limit the training process of the safety recognition model.

[0063] The vehicle light attribute information can be attribute information characterizing the triggering status of the vehicle lights of the obstacle vehicle. There are various types of vehicle light attribute information, and this embodiment does not limit the specific vehicle light attribute information. For example, the vehicle light attribute information may include one or more of the following: left turn signal triggering, right turn signal triggering, brake light triggering, and hazard light triggering. The vehicle door attribute information can be attribute information characterizing the opening and closing status of the vehicle door of the obstacle vehicle. There are various types of vehicle door attribute information, and this embodiment does not limit the specific vehicle door attribute information. For example, the vehicle door attribute information may include one or more of the following: door open, door closed, and door partially open. The lane crossing attribute information can be attribute information characterizing the situation where the obstacle vehicle crosses the lane line of the target vehicle's lane. There are various types of lane crossing attribute information, and this embodiment does not limit the specific lane crossing attribute information. For example, the lane crossing attribute information may include: vehicle not crossing the lane line and / or vehicle crossing the lane line.

[0064] In this embodiment, the information generation device can input the image to be processed into the trained security recognition model, and then use the security recognition model to perform neural network calculations on the image to be processed to obtain security attribute information.

[0065] In some embodiments of this disclosure, the image to be processed is input into a trained security recognition model to obtain security attribute information, including:

[0066] The image to be processed is input into the safety recognition model. If the vehicle category of the obstacle vehicle is within the category filtering range and the vehicle distance is within the distance filtering range, the safety judgment result output by the safety recognition model is used as the safety attribute information. The category filtering range corresponds to the type of safety attribute information, and the distance filtering range corresponds to the type of safety attribute information.

[0067] The category filtering range can be a range of vehicle categories that are retained for filtering vehicles with obstacles at the vehicle category dimension. The vehicle light attribute information can correspond to the first category filtering range, which can be any vehicle category other than tricycles and / or two-wheeled vehicles. The vehicle door attribute information can correspond to the second category filtering range, which can be any vehicle category other than trucks and buses. The line-crossing attribute information can correspond to the third category filtering range, which can be any vehicle category other than tricycles and / or two-wheeled vehicles.

[0068] The distance filtering range can be a range of vehicle distances that are retained and filtered for obstacle vehicles in the vehicle distance dimension. Vehicle light attribute information corresponds to the first distance filtering range, which is a lateral distance within 12 meters and a longitudinal distance within 80 meters. Vehicle door attribute information corresponds to the second distance filtering range, where the lateral and longitudinal distances can be smaller than the first distance filtering range. Line crossing attribute information corresponds to the third distance filtering range, which is a lateral distance within 5 meters and a longitudinal distance within 80 meters.

[0069] The safety judgment result can be the vehicle safety judgment result output from the image to be processed. The safety judgment result can include at least two of the following: headlight judgment result, door judgment result, and line crossing judgment result.

[0070] In this embodiment, the information generation device can input the image to be processed into the safety recognition model and determine whether the vehicle category of the obstacle vehicle is within the category filtering range and whether the distance of the obstacle vehicle is within the distance filtering range. If both are true, the safety judgment result output by the safety recognition model is used as the safety attribute information of the obstacle vehicle. If at least one item is false, the safety judgment result is not assigned to the safety attribute information.

[0071] In the above scheme, multiple tasks are performed through the safety identification model, enabling the simultaneous determination of multiple safety attribute information. This improves the efficiency and real-time performance of safety attribute information determination, further enhancing vehicle safety. Furthermore, when the vehicle category and distance are within the category filtering range, the safety judgment result of the safety identification model is used as the safety attribute information, preventing the safety judgment results of obstacles too far from the target vehicle from affecting the driving of the target vehicle.

[0072] In some embodiments of this disclosure, if the safety attribute information includes door attribute information, and if the vehicle category of the obstacle vehicle is within the category filtering range and the vehicle distance is within the distance filtering range, then the safety judgment result output by the safety recognition model is used as the safety attribute information, including: if the vehicle category information of the obstacle vehicle is within the category filtering range, the vehicle distance is within the distance filtering range, the vehicle speed is within the speed filtering range, and the confidence level of the door judgment result output by the safety recognition model is within the confidence level filtering range, then the door judgment result is used as the door attribute information.

[0073] The speed filtering range can be a range of vehicle speeds used to filter out obstacle vehicles based on vehicle speed. There are various speed filtering ranges, and this embodiment does not limit this range. For example, the speed filtering range can be a range between stationary and near-stationary speeds. The confidence level filtering range can be a range of confidence levels used to filter out obstacle vehicles based on confidence level. There are various confidence level filtering ranges, and this embodiment does not limit this range.

[0074] In this embodiment, based on the door judgment result output by the safety recognition model, the information generation device determines whether the vehicle category of the obstacle vehicle is within the corresponding category filtering range, whether the vehicle distance of the obstacle vehicle is within the corresponding distance filtering range, whether the vehicle speed of the obstacle vehicle is within the corresponding speed filtering range, and whether the confidence level of the door judgment result is within the corresponding confidence level filtering range. If all are true, the door judgment result is assigned to the door attribute information. If at least one item is false, the door judgment result is not assigned to the door attribute information.

[0075] In this embodiment, in addition to judging the car door, the vehicle speed is judged, which realizes the judgment of the relatively dangerous scenario that the obstacle vehicle is about to stop and open the car door. In addition, the confidence of the car door judgment result is judged. Compared with the judgment of the headlights and the line, the judgment of the car door has higher requirements for image recognition. By filtering the confidence, the generation of incorrect car door attribute information is avoided and the accuracy of the car door attribute information is improved.

[0076] Figure 4 A schematic diagram of a security identification model provided in an embodiment of this disclosure, as shown below. Figure 4 As shown, for the headlight judgment results output by the safety recognition model, if the vehicle category of the obstacle vehicle is within the first category filtering range and the vehicle distance of the obstacle vehicle is within the first distance filtering range, the headlight judgment result is assigned to the headlight attribute information. For the door judgment results output by the safety recognition model, if the vehicle category of the obstacle vehicle is within the second category filtering range, the vehicle distance of the obstacle vehicle is within the second distance filtering range, the vehicle speed of the obstacle vehicle is within the speed filtering range, and the confidence level of the door judgment result is within the confidence level filtering range, the door judgment result is assigned to the door attribute information. For the line crossing judgment results output by the safety recognition model, if the vehicle category of the obstacle vehicle is within the third category filtering range and the vehicle distance of the obstacle vehicle is within the third distance filtering range, the door judgment result is assigned to the door attribute information. Thus, after the initial screening of obstacle vehicles, corresponding screening is performed for different types of safety attribute information. This two-stage screening saves computing power and avoids interference from useless information to the user.

[0077] The information generation method provided in this disclosure involves inputting environmental images collected by a data acquisition device into a bird's-eye view model to obtain three-dimensional attribute information of obstacle vehicles around the data acquisition device. The three-dimensional attribute information includes vehicle distances. The environmental images are then filtered based on the vehicle distances to obtain target images corresponding to the obstacle vehicles. Vehicle safety identification is performed on the target images to obtain safety attribute information of the obstacle vehicles. In this disclosure, after processing the environmental images using a bird's-eye view model to obtain the three-dimensional attribute information of obstacle vehicles around the vehicle, the target images corresponding to the obstacle vehicles are further determined based on the vehicle distances in the three-dimensional attribute information. By performing vehicle safety identification on the target images, the safety attribute information of the obstacle vehicles detected by the bird's-eye view model is obtained. This supplements the information obtained from the bird's-eye view model in the safety dimension, improving vehicle driving safety.

[0078] In some embodiments of this disclosure, the three-dimensional attribute information further includes at least one of: vehicle size, vehicle speed, vehicle category, and vehicle orientation. The vehicle size can be the external dimensions of the obstacle vehicle, such as length, width, and height. The vehicle speed can be the travel speed of the obstacle vehicle. The vehicle category can be the vehicle type of the obstacle vehicle, which may include one or more of sedans, buses, and trucks. The vehicle orientation can be the travel direction of the obstacle vehicle.

[0079] Correspondingly, the information generation method also includes: merging three-dimensional attribute information and safety attribute information to obtain vehicle attribute information; wherein, the vehicle attribute information is retrieved through a preset information interface.

[0080] The vehicle attribute information can be a combination of three-dimensional attribute information and safety attribute information. The preset information interface can be a pre-set interface for reading vehicle attribute information. Based on this preset information interface, the vehicle attribute information can be applied in subsequent vehicle trajectory prediction, temporal fusion, and other aspects.

[0081] Figure 5 This is a schematic diagram illustrating the determination of vehicle attribute information provided in an embodiment of this disclosure, such as... Figure 5 As shown, the information generation device can acquire environmental images captured by an environmental camera and input the environmental images into a bird's-eye view model to obtain three-dimensional attribute information. The target image is determined by combining the environmental image and the three-dimensional attribute information. After preprocessing the target image, it is input into a safety recognition model to obtain the safety judgment result output by the safety recognition model. Post-processing is achieved by filtering based on multiple filtering ranges, and the safety judgment result that passes the filtering is determined as the corresponding safety attribute information. The safety attribute information and the three-dimensional attribute information are merged to obtain vehicle attribute information, and a preset information interface is provided so that downstream functions can call the vehicle attribute information.

[0082] The above solution merges three-dimensional attribute information and safety attribute information, and integrates attribute information, making it easier for subsequent downstream functions to call vehicle attribute information.

[0083] The information generation method in this embodiment will be further explained through a specific example. In this embodiment, an environmental image is input into a bird's-eye view model, and the three-dimensional attribute information of the obstacle vehicle is obtained through the bird's-eye view model. In this three-dimensional attribute information, the position of the obstacle vehicle is represented by the coordinates of the eight vertices of the three-dimensional envelope. The obstacle vehicles are pre-filtered according to a preset category range and a preset distance range, filtering out obstacle vehicles outside the preset category range and the preset distance range. The preset category range is a category range obtained by comprehensively considering the first category filtering condition corresponding to the headlight attribute information, the second category filtering condition corresponding to the door attribute information, and the third category filtering condition corresponding to the line crossing attribute information. The preset distance range is a distance range obtained by comprehensively considering the first distance filtering condition corresponding to the headlight attribute information, the second distance filtering condition corresponding to the door attribute information, and the third distance filtering condition corresponding to the line crossing attribute information. The 3D bounding boxes of vehicles output from the bird's-eye view model are transformed into 2D bounding boxes to obtain 2D detection boxes for obstacle vehicles. Target detection boxes within these boxes are then identified. After appropriate expansion, the target detection boxes are used for image matting on both the target image and the expanded target detection boxes, resulting in the image to be processed. This image is then input into a safety recognition model to obtain a safety judgment result. Based on this result, corresponding safety attribute information is assigned. The safety attribute information and 3D attribute information are integrated to obtain vehicle attribute information, which is then passed to downstream systems.

[0084] The information generation method provided in this disclosure, based on a bird's-eye view model, outputs at least two of the more important aspects of an obstacle vehicle's status: headlight status, door status, and lane-crossing status, thereby improving the safety of autonomous driving. Specifically, headlight attribute information aids in handling lane-changing and insertion scenarios by vehicles ahead in autonomous driving. Door attribute information allows for setting the door status of the obstacle vehicle's 3D model within the Environmental Interpretation and Detection (EID) system, enabling users to intuitively observe whether the obstacle vehicle's doors are open, thus increasing driving safety. Lane-crossing attribute information can be used to determine whether an obstacle vehicle is changing lanes, enabling the prediction of the obstacle vehicle's trajectory. Based on these safety attribute information, the needs of the target vehicle in different tasks can be met. Furthermore, this information generation method, based on the output of a bird's-eye view model, has low addition time, requires minimal modification, is easy to deploy in real vehicles, and simultaneously outputs multiple safety attribute information, improving vehicle safety.

[0085] Figure 6 A schematic diagram of the structure of an information generation apparatus provided in an embodiment of this disclosure is shown.

[0086] In some embodiments of this disclosure, Figure 6 The information generation device shown can be executed by an electronic device or a server. The electronic device can be, but is not limited to, mobile terminals such as in-vehicle terminals, and fixed terminals such as vehicle domain controllers. The server can be a server cluster or a cloud server.

[0087] like Figure 6 As shown, the information generation device 600 may include: an input module 601, a filtering module 602, and a security module 603.

[0088] The input module 601 is used to input the environmental image collected by the acquisition device into the bird's-eye view model to obtain the three-dimensional attribute information of the obstacle vehicles around the acquisition device; wherein, the three-dimensional attribute information includes the vehicle distance;

[0089] The filtering module 602 is used to filter the environmental image according to the vehicle distance to obtain the target image corresponding to the obstacle vehicle;

[0090] The safety module 603 is used to perform vehicle safety identification on the target image and obtain the safety attribute information of the obstacle vehicle.

[0091] Optionally, the filtering module 602 is used for:

[0092] The environmental images containing the obstructing vehicles were identified as candidate images.

[0093] If there are multiple candidate images and the vehicle distance is within a distance threshold range, then the largest detection box in the multiple detection boxes corresponding to the multiple candidate images is determined as the target detection box, and the candidate image containing the target detection box is determined as the target image.

[0094] Optionally, the filtering module 602 is further configured to:

[0095] If there are multiple candidate images and the vehicle distance is not within the distance threshold range, the candidate images are filtered according to the camera orientation priority corresponding to the vehicle distance to obtain the target image; wherein, the forward-looking orientation takes precedence over the side-forward-looking orientation in the camera orientation priority.

[0096] Optionally, the security module 603 includes:

[0097] The cropping submodule is used to crop the portion of the target image located within the target detection box to obtain the image to be processed.

[0098] The safety submodule is used to input the image to be processed into the trained safety recognition model to obtain safety attribute information; wherein, the safety attribute information includes at least two of the following: vehicle headlight attribute information, vehicle door attribute information, and line crossing attribute information.

[0099] Optionally, the security submodule is used for:

[0100] The image to be processed is input into the safety recognition model. If the vehicle category of the obstacle vehicle is within the category filtering range and the vehicle distance is within the distance filtering range, the safety judgment result output by the safety recognition model is used as the safety attribute information. The category filtering range corresponds to the type of the safety attribute information, and the distance filtering range corresponds to the type of the safety attribute information.

[0101] Optionally, if the safety attribute information includes door attribute information, and if the vehicle category of the obstacle vehicle is within the category filtering range and the vehicle distance is within the distance filtering range, then the safety judgment result output by the safety recognition model is used as the safety attribute information, including:

[0102] If the vehicle category information of the obstacle vehicle is within the category filtering range, the vehicle distance is within the distance filtering range, the vehicle speed is within the speed filtering range, and the confidence level of the door judgment result output by the safety recognition model is within the confidence level filtering range, then the door judgment result will be used as the door attribute information.

[0103] Optionally, the device further includes:

[0104] An extension module is used to expand the target detection box to obtain an extended detection box before cropping the portion of the target image located within the target detection box, and to update the target detection box with the extended detection box.

[0105] Optionally, the three-dimensional attribute information may further include at least one of the following: vehicle size, vehicle speed, vehicle category, and vehicle orientation;

[0106] Accordingly, the device also includes:

[0107] The merging module is used to merge the three-dimensional attribute information and the safety attribute information to obtain vehicle attribute information; wherein the vehicle attribute information is retrieved through a preset information interface.

[0108] The information generation apparatus provided in this disclosure can execute the information generation method provided in any embodiment of this disclosure, and has the corresponding functional modules and beneficial effects of executing the method.

[0109] Figure 7A schematic diagram of the hardware circuit structure of an information generation device provided in an embodiment of this disclosure is shown.

[0110] like Figure 7 As shown, the information generation device 700 may include a controller 701 and a memory 702 storing computer program instructions.

[0111] Specifically, the controller 701 may include a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.

[0112] Memory 702 may include a large-capacity storage for information or instructions. For example, and not limitingly, memory 702 may include a hard disk drive (HDD), a floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or a Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 702 may include removable or non-removable (or fixed) media. Where appropriate, memory 702 may be internal or external to the integrated gateway device. In a particular embodiment, memory 702 is a non-volatile solid-state memory. In a particular embodiment, memory 702 includes read-only memory (ROM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (Electrically Programmable ROM, EPROM), an electrically erasable programmable PROM (EEPROM), an electrically alterable ROM (EAROM), or flash memory, or a combination of two or more of these.

[0113] The controller 701 reads and executes computer program instructions stored in the memory 702 to perform the steps of the information generation method provided in the embodiments of this disclosure.

[0114] In one example, the information generation device 700 may further include a transceiver 703 and a bus 704. Wherein, as... Figure 7 As shown, the controller 701, memory 702 and transceiver 703 are connected via bus 704 and communicate with each other.

[0115] Bus 704 includes hardware, software, or both. For example, and not limitingly, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industrial Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a MicroChannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local Bus (VLB) bus, or other suitable buses, or a combination of two or more of these. Where appropriate, bus 704 may include one or more buses. Although specific buses are described and illustrated in the embodiments of this application, this application considers any suitable bus or interconnection.

[0116] The following are embodiments of a computer-readable storage medium provided in this disclosure. This computer-readable storage medium and the information generation methods of the above embodiments belong to the same inventive concept. For details not described in detail in the embodiments of the computer-readable storage medium, please refer to the embodiments of the above information generation methods.

[0117] This embodiment provides a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform an information generation method.

[0118] Of course, the computer-executable instructions provided in the embodiments of this disclosure are not limited to the above-described method operations, but can also perform related operations in the information generation method provided in any embodiment of this disclosure.

[0119] Based on the above description of the implementation methods, those skilled in the art can clearly understand that this disclosure can be implemented using software and necessary general-purpose hardware, and of course, it can also be implemented using hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this disclosure, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ROM), random access memory (RAM), flash memory, hard disk, or optical disk, etc., including several instructions to cause a computer cloud platform (which may be a personal computer, server, or network cloud platform, etc.) to execute the information generation methods provided in the various embodiments of this disclosure.

[0120] This disclosure also provides a vehicle, including at least one of the following: the aforementioned information generation device; the aforementioned electronic device; and the aforementioned computer-readable storage medium.

[0121] Note that the above description is merely a preferred embodiment and the technical principles employed in this disclosure. Those skilled in the art will understand that this disclosure is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of this disclosure. Therefore, although this disclosure has been described in detail through the above embodiments, it is not limited to the above embodiments. Many other equivalent embodiments may be included without departing from the concept of this disclosure, and the scope of this disclosure is determined by the scope of the appended claims.

Claims

1. An information generation method, characterized in that, include: The environmental images collected by the acquisition device are input into the bird's-eye view model to obtain the three-dimensional attribute information of obstacle vehicles around the acquisition device; wherein, the three-dimensional attribute information includes the vehicle distance; The environmental images are filtered based on the vehicle distance to obtain the target image corresponding to the obstacle vehicle; Vehicle safety identification is performed on the target image to obtain the safety attribute information of the obstacle vehicle.

2. The method according to claim 1, characterized in that, The step of filtering the environmental image based on the vehicle distance to obtain the target image corresponding to the obstacle vehicle includes: The environmental images containing the obstructing vehicles were identified as candidate images. If there are multiple candidate images and the vehicle distance is within a distance threshold range, then the largest detection box in the multiple detection boxes corresponding to the multiple candidate images is determined as the target detection box, and the candidate image containing the target detection box is determined as the target image.

3. The method according to claim 2, characterized in that, The method further includes: If there are multiple candidate images and the vehicle distance is not within the distance threshold range, the candidate images are filtered according to the camera orientation priority corresponding to the vehicle distance to obtain the target image; wherein, the forward-looking orientation takes precedence over the side-forward-looking orientation in the camera orientation priority.

4. The method according to claim 1, characterized in that, The step of performing vehicle safety identification on the target image to obtain the safety attribute information of the obstacle vehicle includes: The portion of the target image located within the target detection box is cropped to obtain the image to be processed; The image to be processed is input into the trained security recognition model to obtain security attribute information; wherein, the security attribute information includes at least two of the following: vehicle headlight attribute information, vehicle door attribute information, and line crossing attribute information.

5. The method according to claim 4, characterized in that, The step of inputting the image to be processed into the trained security recognition model to obtain security attribute information includes: The image to be processed is input into the safety recognition model. If the vehicle category of the obstacle vehicle is within the category filtering range and the vehicle distance is within the distance filtering range, the safety judgment result output by the safety recognition model is used as the safety attribute information. The category filtering range corresponds to the type of the safety attribute information, and the distance filtering range corresponds to the type of the safety attribute information.

6. The method according to claim 5, characterized in that, If the safety attribute information includes door attribute information, and if the vehicle category of the obstacle vehicle is within the category filtering range and the vehicle distance is within the distance filtering range, then the safety judgment result output by the safety recognition model is used as the safety attribute information, including: If the vehicle category information of the obstacle vehicle is within the category filtering range, the vehicle distance is within the distance filtering range, the vehicle speed is within the speed filtering range, and the confidence level of the door judgment result output by the safety recognition model is within the confidence level filtering range, then the door judgment result will be used as the door attribute information.

7. The method according to claim 4, characterized in that, Before cropping the portion of the target image located within the target detection box, the method further includes: The target detection box is expanded to obtain an extended detection box, and the target detection box is updated to the extended detection box.

8. The method according to claim 1, characterized in that, The three-dimensional attribute information also includes at least one of the following: vehicle size, vehicle speed, vehicle category, and vehicle orientation; Accordingly, the method further includes: The three-dimensional attribute information and the safety attribute information are merged to obtain vehicle attribute information; wherein, the vehicle attribute information is retrieved through a preset information interface.

9. An information generation method, characterized in that, include: The input module is used to input environmental images collected by the acquisition device into a bird's-eye view model to obtain three-dimensional attribute information of obstacle vehicles around the acquisition device; wherein, the three-dimensional attribute information includes vehicle distance; The filtering module is used to filter the environmental image based on the vehicle distance to obtain the target image corresponding to the obstacle vehicle; The safety module is used to perform vehicle safety identification on the target image and obtain the safety attribute information of the obstacle vehicle.

10. An electronic device, characterized in that, The electronic device includes: Processor and memory; The processor executes the steps of the method as described in any one of claims 1 to 8 by invoking programs or instructions stored in the memory.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a program or instructions that cause a computer to perform the steps of the method as described in any one of claims 1 to 8.

12. A vehicle, characterized in that, Includes at least one of the following: The key point matching device as described in claim 9 above; The electronic device according to claim 10; The computer-readable storage medium as described in claim 11.

Images