Visual identification method and device for pilotless automobile and medium

Abstract

The invention relates to the artificial intelligence technology, and provides a pilotless automobile visual identification method and device and a medium, and the method comprises the steps: carrying out the environment adaptability enhancement processing of a collected initial environment image of a target automobile, and obtaining an environment enhancement image; extracting image features of the environment enhanced image, and then performing feature classification of the image features to obtain classification features of the image features; generating a movement track of a change feature in the classification features; generating a traffic scene of the target automobile according to fixed features in the classification features and a preset energy function; and generating a path plan of the target automobile according to the movement track and the traffic scene, performing instruction conversion on the path plan to obtain a control instruction of the target automobile, and performing driving operation on the target automobile by using the control instruction. According to the invention, the visual identification accuracy of the pilotless automobile can be improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a method, device and medium for visual recognition of an unmanned vehicle. Background Art

[0002] With the gradual promotion of driverless technology, ensuring the reliability of this technology is the key to protecting the safety of all traffic participants. Accurately identifying road signs, pedestrians, other vehicles and obstacles is the basis for ensuring safe driving.

[0003] The technical level of sensors that unmanned driving systems rely on (such as cameras, radars, lidar, etc.) is constantly improving, and the correct use of sensor data is particularly important. At the same time, the driving environment is very complex and changeable. Early visual recognition systems may find it difficult to cope with various emergencies, such as weather changes, road sign obstructions, etc. Summary of the invention

[0004] The present invention provides a method, device, equipment and computer-readable storage medium for visual recognition of an unmanned vehicle, the main purpose of which is to solve the problem of low accuracy in visual recognition of an unmanned vehicle.

[0005] To achieve the above object, the present invention provides a driverless car visual recognition method, comprising:

[0006] Collecting an initial environment image of a target vehicle, performing environmental adaptability enhancement processing on the initial environment image, and obtaining an environmental enhanced image of the target vehicle;

[0007] Extracting image features of the environment enhanced image;

[0008] Using a pre-trained convolutional neural network model to perform feature classification on the image features to obtain classification features of the image features;

[0009] Predicting the trajectory of the change feature in the classification feature to obtain the motion trajectory of the change feature;

[0010] The traffic scene of the target vehicle is generated according to the fixed features in the classification features and a preset energy function, wherein the preset energy function is:

[0011]

[0012] Where E(X) is the energy function, N is the total number of static environmental elements, and X i is the state of the ith static environmental element, X j is the state of the jth static environmental element, h i is the tendency of the i-th static environmental element itself, J ijis the interaction intensity between the ith static environmental element and the jth static environmental element;

[0013] A path plan of the target vehicle is generated according to the motion trajectory and the traffic scene, the path plan is converted into instructions to obtain a control instruction of the target vehicle, and the control instruction is used to drive the target vehicle.

[0014] Optionally, performing environmental adaptability enhancement processing on the initial environmental image to obtain an environmental enhanced image of the target vehicle includes:

[0015] The initial environment image is corrected using a preset affine transformation matrix to obtain a corrected image of the initial environment image, wherein the preset affine transformation matrix is:

[0016]

[0017] Among them, a and d represent the scaling factors along the x-axis and y-axis respectively, b and c represent the shearing factors along the x-axis and y-axis respectively, (t x ,t y ) is the translation vector, A is the affine transformation matrix;

[0018] The contrast of the corrected image is adjusted, and the brightness of the corrected image after the contrast adjustment is adjusted to obtain an environment enhanced image of the target vehicle.

[0019] Optionally, extracting image features of the environment enhanced image includes:

[0020] Dividing the environment enhanced image into regions to obtain a region image of the environment enhanced image;

[0021] Identifying regional objects in the regional image one by one;

[0022] Marking feature points of objects in the region to obtain key feature points of objects in the region;

[0023] The image features of the environment enhanced image are generated according to the key feature points.

[0024] Optionally, the using a pre-trained convolutional neural network model to perform feature classification on the image features to obtain classification features of the image features includes:

[0025] Inputting the image features into a pre-trained convolutional neural network model, and performing multi-layer convolution on the image features using the pre-trained convolutional neural network model to obtain convolution features of the image features;

[0026] Performing a pooling operation on the convolutional features to obtain pooled features of the convolutional features;

[0027] A probability distribution of the pooled features is generated, and classification features of the image features are determined according to the probability distribution.

[0028] Optionally, the classification features include: variable features and fixed features, wherein the variable features include: pedestrians and vehicles, and the fixed features include: buildings, trees and road signs.

[0029] Optionally, performing trajectory prediction on the change feature in the classification feature to obtain the motion trajectory of the change feature includes:

[0030] Obtaining the initial state of the change feature in the current frame;

[0031] determining a movement trend of the change characteristic;

[0032] A motion trajectory of the changing feature is generated according to the initial state and the motion trend.

[0033] Optionally, generating the traffic scene of the target vehicle according to the fixed features in the classification features and a preset energy function includes:

[0034] The static environment elements of the current frame are determined according to the fixed features in the classification features and a preset energy function, wherein the preset energy function is:

[0035]

[0036] Where E(X) is the energy function, N is the total number of static environmental elements, and X i is the state of the ith static environmental element, X j is the state of the jth static environmental element, h i is the tendency of the i-th static environmental element itself, J ij is the interaction intensity between the ith static environmental element and the jth static environmental element;

[0037] Performing environmental modeling according to the static environmental elements to obtain an initial environmental model of the target vehicle;

[0038] The initial environment model is enhanced in detail to obtain the traffic scene of the target vehicle.

[0039] Optionally, generating the path planning of the target vehicle according to the motion trajectory and the traffic scene includes:

[0040] Generate obstacles to be avoided by the target vehicle according to the motion trajectory and the traffic scene;

[0041] Generating a safe obstacle avoidance path for the target vehicle according to the obstacle to be avoided;

[0042] Generate a path plan for the target vehicle based on the shortest path in the safe obstacle avoidance path.

[0043] In order to solve the above problems, the present invention further provides a visual recognition device for an unmanned vehicle, the device comprising:

[0044] An image processing module is used to collect an initial environment image of a target vehicle, perform environmental adaptability enhancement processing on the initial environment image, and obtain an environmental enhanced image of the target vehicle;

[0045] A feature extraction module, used to extract image features of the environment enhanced image;

[0046] A feature classification module, used to perform feature classification on the image features using a pre-trained convolutional neural network model to obtain classification features of the image features;

[0047] A trajectory prediction module, used for performing trajectory prediction on the change feature in the classification feature to obtain the motion trajectory of the change feature;

[0048] The traffic scene generation module is used to generate the traffic scene of the target vehicle according to the fixed features in the classification features and a preset energy function, wherein the preset energy function is:

[0049]

[0050] Where E(X) is the energy function, N is the total number of static environmental elements, and X i is the state of the ith static environmental element, X j is the state of the jth static environmental element, h i is the tendency of the i-th static environmental element itself, J ij is the interaction intensity between the ith static environmental element and the jth static environmental element;

[0051] The path planning module is used to generate the path planning of the target vehicle according to the motion trajectory and the traffic scene, perform instruction conversion on the path planning, obtain the control instruction of the target vehicle, and use the control instruction to drive the target vehicle.

[0052] In order to solve the above problems, the present invention also provides a computer-readable storage medium, in which at least one computer program is stored. The at least one computer program is executed by a processor in an electronic device to implement the above-mentioned unmanned vehicle visual recognition method.

[0053] The embodiment of the present invention improves the image quality by collecting the initial environment image and performing environmental adaptability enhancement processing to reduce the impact of lighting and weather changes. Then, the key features of these enhanced images are extracted, and the feature classification is performed using a pre-trained convolutional neural network model to automatically learn the ability to distinguish different objects. Subsequently, the trajectory prediction of dynamic features is performed to help the system respond in advance. At the same time, traffic scenes are generated based on fixed features and preset energy functions, and the vehicle's understanding of the environment is optimized by minimizing the energy function. Finally, path planning is generated based on the motion trajectory and traffic scenes, and converted into specific driving instructions to control the vehicle. The entire process ensures that the vehicle can accurately understand the environment and drive safely by gradually improving the image quality and using advanced machine learning technology. Therefore, the unmanned vehicle visual recognition method, device and medium proposed in the present invention can solve the problem of low accuracy of unmanned vehicle visual recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 A schematic diagram of a process flow of a driverless car visual recognition method provided by an embodiment of the present invention;

[0055] Figure 2 A functional module diagram of a driverless car visual recognition device provided by one embodiment of the present invention.

[0056] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0057] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0058] The embodiment of the present application provides a method for visual recognition of an unmanned vehicle. The execution subject of the method for visual recognition of an unmanned vehicle includes but is not limited to at least one of the electronic devices such as a server and a terminal that can be configured to execute the method provided in the embodiment of the present application. In other words, the method for visual recognition of an unmanned vehicle can be executed by software or hardware installed on a terminal device or a server device, and the software can be a blockchain platform. The server includes but is not limited to: a single server, a server cluster, a cloud server or a cloud server cluster, etc. The server can be an independent server, or it can be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (Content Delivery Network, CDN), and big data and artificial intelligence platforms.

[0059] Reference Figure 1FIG. 1 is a flow chart of a method for visually identifying an unmanned vehicle provided by an embodiment of the present invention. In this embodiment, the method for visually identifying an unmanned vehicle includes:

[0060] S1. Collecting an initial environment image of a target vehicle, performing environmental adaptability enhancement processing on the initial environment image, and obtaining an environmental enhanced image of the target vehicle.

[0061] In an embodiment of the present invention, the acquisition of the initial environmental image of the target vehicle refers to the use of various vehicle-mounted sensors to collect environmental information around the driverless vehicle. This step is the basis for the driverless vehicle to perceive the world around it and provides necessary data for subsequent image processing and decision making.

[0062] In detail, various sensors are installed on the driverless car to ensure that they can cover all important areas around the vehicle.

[0063] Furthermore, when the vehicle is started, various sensors begin to work, capturing images and other types of data of the surrounding environment.

[0064] In the embodiment of the present invention, the step of performing environmental adaptability enhancement processing on the initial environmental image to obtain the environmental enhanced image of the target vehicle includes:

[0065] The initial environment image is corrected using a preset affine transformation matrix to obtain a corrected image of the initial environment image, wherein the preset affine transformation matrix is:

[0066]

[0067] Among them, a and d represent the scaling factors along the x-axis and y-axis respectively, b and c represent the shearing factors along the x-axis and y-axis respectively, (t x ,t y ) is the translation vector, A is the affine transformation matrix;

[0068] The contrast of the corrected image is adjusted, and the brightness of the corrected image after the contrast adjustment is adjusted to obtain an environment enhanced image of the target vehicle.

[0069] Specifically, performing environmental adaptability enhancement processing on the initial environmental image refers to improving the quality of the image through a series of image processing techniques to make it more suitable for subsequent feature extraction and recognition tasks.

[0070] In detail, the initial environment image is corrected using a preset affine transformation matrix to eliminate geometric distortion in the image and improve image quality.

[0071] In detail, contrast adjustment is to enhance the details in the image, making the difference between light and dark in the image more obvious. This can be achieved through a variety of methods, such as histogram equalization, gamma correction or using contrast stretching techniques.

[0072] In detail, brightness adjustment is to ensure the overall visibility of the image so that the image can maintain good readability under different lighting conditions, which can be achieved by increasing or decreasing the overall brightness value of the image.

[0073] In detail, affine transformation is a linear transformation that can rotate, scale, shear, and translate an image without changing the parallel line relationship in the image, which is very useful for correcting image deformation caused by the camera installation angle or vehicle posture.

[0074] Furthermore, the scaling factor determines how much the image is enlarged or reduced on the corresponding axis, the shear factor can be used to change the tilt of the image, and the translation vector determines the displacement of the image in the x-axis and y-axis directions.

[0075] S2. Extracting image features of the environment enhanced image.

[0076] In an embodiment of the present invention, extracting image features of the environment enhanced image includes:

[0077] Dividing the environment enhanced image into regions to obtain a region image of the environment enhanced image;

[0078] Identifying regional objects in the regional image one by one;

[0079] Marking feature points of objects in the region to obtain key feature points of objects in the region;

[0080] The image features of the environment enhanced image are generated according to the key feature points.

[0081] In detail, region segmentation refers to dividing an image into multiple sub-regions or paragraphs. This step can help focus on different parts of the image, making it easier to detect and identify specific objects. There are many ways to segment the region, which can be processed based on thresholds.

[0082] In detail, object recognition is to determine whether there is a target object of interest in each sub-region. This step is usually performed based on machine learning or deep learning methods, and can be based on deep learning models such as YOLO, SSD, Faster R-CNN, etc.

[0083] In detail, feature points refer to points with unique properties in an image, which are very important for describing the shape, position and posture of an object. These feature points can be corner points, edge points, etc.

[0084] In detail, feature description generates a descriptor for each detected feature point, which can describe the local information around the point. These descriptors can be used to compare the similarities between different images.

[0085] S3. Using a pre-trained convolutional neural network model to perform feature classification on the image features to obtain classification features of the image features.

[0086] In an embodiment of the present invention, the method of performing feature classification on the image features using a pre-trained convolutional neural network model to obtain classification features of the image features includes:

[0087] Inputting the image features into a pre-trained convolutional neural network model, and performing multi-layer convolution on the image features using the pre-trained convolutional neural network model to obtain convolution features of the image features;

[0088] Performing a pooling operation on the convolutional features to obtain pooled features of the convolutional features;

[0089] A probability distribution of the pooled features is generated, and classification features of the image features are determined according to the probability distribution.

[0090] In detail, the image is input into CNN, and the image is filtered through multiple convolutional layers to obtain the convolution features of the image, which are usually abstract representations of local features such as edges and textures in the image. It reduces the feature dimension and retains important information while improving the robustness of the model.

[0091] In detail, after each convolution layer, there is usually a pooling layer (such as max pooling or average pooling), which reduces the size of the convolution feature map to obtain pooled features, which have better translation invariance and reduce computational complexity.

[0092] In detail, in the last few layers of CNN, there will be fully connected layers, which are used to map the pooled features into a low-dimensional space, in which each output node represents a category and outputs a probability vector indicating the probability of each category.

[0093] In detail, based on the above probability distribution, it is determined which category the image belongs to, and the category with the highest probability can be selected as the final classification result to obtain the classification feature, that is, the specific object category contained in the image.

[0094] In detail, the classification features include: variable features and fixed features, wherein the variable features include: pedestrians and vehicles, and the fixed features include: buildings, trees and road signs.

[0095] In detail, classification features are further subdivided into variable features and fixed features. The variable features refer to objects that may move in the image, such as pedestrians and vehicles; while the fixed features refer to objects with relatively fixed positions, such as buildings, trees, and road signs.

[0096] S4. Predicting the trajectory of the change feature in the classification feature to obtain the motion trajectory of the change feature.

[0097] In the embodiment of the present invention, the step of performing trajectory prediction on the change feature in the classification feature to obtain the motion trajectory of the change feature includes:

[0098] Obtaining the initial state of the change feature in the current frame;

[0099] determining a movement trend of the change characteristic;

[0100] A motion trajectory of the changing feature is generated according to the initial state and the motion trend.

[0101] Specifically, the initial position, speed and other state information of the changing feature (such as a pedestrian or a vehicle) in the current image frame are obtained.

[0102] In detail, the position coordinates of the changing features are extracted from the classification results to obtain the initial states of the changing features, such as the position coordinates and the speed.

[0103] Furthermore, to analyze the motion pattern of the changing features and predict their possible future motion direction and speed changes, it is usually necessary to analyze the position changes of the changing features in multiple consecutive frames of images to estimate their motion trends. Kalman filters, particle filters and other methods can be used to perform trend prediction and obtain the motion trend of the changing features, including future speed and direction.

[0104] In detail, generating the motion trajectory of the changing feature according to the initial state and the motion trend is to generate the motion trajectory within a future period of time by combining the initial state of the changing feature and the predicted motion trend.

[0105] In detail, kinematic equations or other mathematical models are used to simulate the motion path of the moving feature. For example, if uniform linear motion is assumed, simple linear interpolation can be used to predict future coordinates to obtain a series of future position coordinates of the moving feature, that is, its motion trajectory.

[0106] S5. Generate a traffic scene of the target vehicle according to the fixed features in the classification features and a preset energy function.

[0107] In the embodiment of the present invention, the generating of the traffic scene of the target vehicle according to the fixed features in the classification features and the preset energy function includes:

[0108] The static environment elements of the current frame are determined according to the fixed features in the classification features and a preset energy function, wherein the preset energy function is:

[0109]

[0110] Where E(X) is the energy function, N is the total number of static environmental elements, and X i is the state of the ith static environmental element, X j is the state of the jth static environmental element, h i is the tendency of the i-th static environmental element itself, J ij is the interaction intensity between the ith static environmental element and the jth static environmental element;

[0111] Performing environmental modeling according to the static environmental elements to obtain an initial environmental model of the target vehicle;

[0112] The initial environment model is enhanced in detail to obtain the traffic scene of the target vehicle.

[0113] In detail, the energy function E(X) is a mathematical model used to describe the interaction between static environmental elements.

[0114] Furthermore, it consists of two parts: the first part∑ (i,j)∈N J ij X i X j Describes the interaction strength between different static environmental elements; the second part∑ i h i X i Describes the tendency of each static environmental element itself.

[0115] In detail, based on a given energy function, an optimization algorithm can be used to find the state combination X that minimizes E(X), which usually means finding the most stable static environment configuration.

[0116] In detail, after the static environment elements are determined, the next step is to build the environment model, create a scene graph or mesh model containing all relevant elements, and map the state of each element to the corresponding position in the model.

[0117] In detail, the initial environment model includes all static environmental elements around the target car, such as road boundaries, road signs, buildings, etc.

[0118] In detail, in order to make the model more realistic and practical, more details need to be added, including adding dynamic elements (such as other vehicles, pedestrians), as well as more refined environmental elements (such as trees, street lights, etc.). The final output is a complete traffic scene containing static and dynamic elements, which can be used to simulate and predict the behavior of the target car.

[0119] S6. Generate a path plan for the target vehicle according to the motion trajectory and the traffic scene, perform command conversion on the path plan to obtain a control command for the target vehicle, and use the control command to drive the target vehicle.

[0120] In an embodiment of the present invention, the generating of the path planning of the target vehicle according to the motion trajectory and the traffic scene includes:

[0121] Generate obstacles to be avoided by the target vehicle according to the motion trajectory and the traffic scene;

[0122] Generating a safe obstacle avoidance path for the target vehicle according to the obstacle to be avoided;

[0123] Generate a path plan for the target vehicle based on the shortest path in the safe obstacle avoidance path.

[0124] In detail, the result of path planning may contain some discontinuous or overly sharp turns, which are not suitable for direct control of the car. Therefore, the planned path needs to be smoothed first to ensure that the path is more continuous and meets the actual driving conditions.

[0125] In detail, based on the smoothed path, a series of position points and directions in time series, namely trajectories, are generated. These trajectory points contain not only position information but also dynamic information such as speed and acceleration, which are used to guide the car how to drive along this path.

[0126] In detail, the control instructions can be longitudinal control, including: controlling the speed and acceleration of the car, for example, accelerating, decelerating, maintaining a constant speed, etc.; lateral control, including: controlling the direction of the car, for example, turning left, turning right, keeping straight, etc.

[0127] In detail, the calculated control instructions are sent to the various actuators of the car (such as motors, brakes, steering systems, etc.) to achieve actual control of the car. During the execution process, the car's status (such as position, speed, etc.) needs to be continuously fed back and the control instructions are adjusted according to the actual situation to ensure that the car can accurately track the predetermined trajectory.

[0128] In an embodiment of the present invention, the path planning is converted into a command to obtain a control command for the target vehicle, and the driving operation of the target vehicle using the control command includes:

[0129] Assume that a path planning result from point A to point B has been obtained. Now it needs to be converted into specific control instructions: use curve fitting or other smoothing algorithms to ensure that the path has no unnecessary bends or mutations. According to the smoothed path, generate a series of position points in the time series and assign appropriate speed and acceleration values ​​to each point. Interpolation methods may be used to refine the points on the path. If the target speed is higher than the current speed, an acceleration instruction is issued. If the target speed is lower than the current speed, a deceleration instruction is issued. If the target speed is close to the current speed, the current speed is maintained. If you need to turn left, a left turn instruction is issued. If you need to turn right, a right turn instruction is issued. If you need to keep driving in a straight line, a keep driving in a straight line instruction is issued.

[0130] Furthermore, it is also possible to consider updating the path planning and control instructions in real time due to changes in road conditions, ensuring that all control instructions are safe, avoiding any potential risks, minimizing unnecessary acceleration and deceleration, and improving passenger comfort. In this way, the results of path planning can be converted into specific control instructions, thereby achieving effective control of the target vehicle.

[0131] The embodiment of the present invention improves the image quality by collecting the initial environment image and performing environmental adaptability enhancement processing to reduce the impact of lighting and weather changes. Then, the key features of these enhanced images are extracted, and the feature classification is performed using a pre-trained convolutional neural network model to automatically learn the ability to distinguish different objects. Subsequently, the trajectory prediction of dynamic features is performed to help the system respond in advance. At the same time, traffic scenes are generated based on fixed features and preset energy functions, and the vehicle's understanding of the environment is optimized by minimizing the energy function. Finally, path planning is generated based on the motion trajectory and traffic scenes, and converted into specific driving instructions to control the vehicle. The entire process ensures that the vehicle can accurately understand the environment and drive safely by gradually improving the image quality and using advanced machine learning technology. Therefore, the unmanned vehicle visual recognition method proposed in the present invention can solve the problem of low accuracy of unmanned vehicle visual recognition.

[0132] like Figure 2 , which is a functional module diagram of a driverless car visual recognition device provided by one embodiment of the present invention.

[0133] The unmanned vehicle visual recognition device 100 of the present invention can be installed in an electronic device. According to the functions to be implemented, the unmanned vehicle visual recognition device 100 can include an image processing module 101, a feature extraction module 10, a feature classification module 103, a trajectory prediction module 104, a traffic scene generation module 105 and a path planning module 106. The module of the present invention can also be called a unit, which refers to a series of computer program segments that can be executed by an electronic device processor and can complete fixed functions, which are stored in the memory of the electronic device.

[0134] In this embodiment, the functions of each module / unit are as follows:

[0135] The image processing module 101 is used to collect an initial environment image of a target vehicle, perform environmental adaptability enhancement processing on the initial environment image, and obtain an environmental enhanced image of the target vehicle;

[0136] The feature extraction module 102 is used to extract image features of the environment enhanced image;

[0137] The feature classification module 103 is used to perform feature classification on the image features using a pre-trained convolutional neural network model to obtain classification features of the image features;

[0138] The trajectory prediction module 104 is used to perform trajectory prediction on the change feature in the classification feature to obtain the motion trajectory of the change feature;

[0139] The traffic scene generation module 105 is used to generate the traffic scene of the target vehicle according to the fixed features in the classification features and a preset energy function, wherein the preset energy function is:

[0140]

[0141] Where E(X) is the energy function, N is the total number of static environmental elements, and X i is the state of the ith static environmental element, X j is the state of the jth static environmental element, h i is the tendency of the i-th static environmental element itself, J ij is the interaction intensity between the ith static environmental element and the jth static environmental element;

[0142] The path planning module 106 is used to generate a path planning of the target vehicle according to the motion trajectory and the traffic scene, perform instruction conversion on the path planning, obtain a control instruction of the target vehicle, and use the control instruction to drive the target vehicle.

[0143] In detail, each module described in the driverless car visual recognition device 100 in the embodiment of the present invention is used in the same manner as described above. Figure 1 The technical means are the same as the unmanned vehicle visual recognition method described in, and can produce the same technical effects, so I will not go into details here.

[0144] The present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor of an electronic device, the computer program can implement:

[0145] Collecting an initial environment image of a target vehicle, performing environmental adaptability enhancement processing on the initial environment image, and obtaining an environmental enhanced image of the target vehicle;

[0146] Extracting image features of the environment enhanced image;

[0147] Using a pre-trained convolutional neural network model to perform feature classification on the image features to obtain classification features of the image features;

[0148] Predicting the trajectory of the change feature in the classification feature to obtain the motion trajectory of the change feature;

[0149] The traffic scene of the target vehicle is generated according to the fixed features in the classification features and a preset energy function, wherein the preset energy function is:

[0150]

[0151] Where E(X) is the energy function, N is the total number of static environmental elements, and X i is the state of the ith static environmental element, X j is the state of the jth static environmental element, h i is the tendency of the i-th static environmental element itself, J ij is the interaction intensity between the ith static environmental element and the jth static environmental element;

[0152] A path plan of the target vehicle is generated according to the motion trajectory and the traffic scene, the path plan is converted into instructions to obtain a control instruction of the target vehicle, and the control instruction is used to drive the target vehicle.

[0153] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the modules is only a logical function division, and there may be other division methods in actual implementation.

[0154] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0155] In addition, each functional module in each embodiment of the present invention may be integrated into one processing unit, each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional modules.

[0156] It is obvious to those skilled in the art that the present invention is not limited to the details of the above exemplary embodiments, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0157] The blockchain referred to in this invention is a new application mode of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, encryption algorithm, etc. Blockchain is essentially a decentralized database, a string of data blocks generated by cryptographic methods. Each data block contains a batch of network transaction information, which is used to verify the validity of its information (anti-counterfeiting) and generate the next block. Blockchain can include the underlying blockchain platform, platform product service layer, and application service layer.

[0158] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0159] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present invention.

Claims

1. A method for visual recognition of an unmanned vehicle, characterized in that: The method comprises: Collecting an initial environment image of a target vehicle, performing environmental adaptability enhancement processing on the initial environment image, and obtaining an environmental enhanced image of the target vehicle; Extracting image features of the environment enhanced image; Using a pre-trained convolutional neural network model to perform feature classification on the image features to obtain classification features of the image features; Predicting the trajectory of the change feature in the classification feature to obtain the motion trajectory of the change feature; The traffic scene of the target vehicle is generated according to the fixed features in the classification features and a preset energy function, wherein the preset energy function is: Where E(X) is the energy function, N is the total number of static environmental elements, and X i is the state of the ith static environmental element, X j is the state of the jth static environmental element, h i is the tendency of the i-th static environmental element itself, J ij is the interaction intensity between the ith static environmental element and the jth static environmental element; A path plan of the target vehicle is generated according to the motion trajectory and the traffic scene, the path plan is converted into instructions to obtain a control instruction of the target vehicle, and the control instruction is used to drive the target vehicle.

2. The driverless car visual recognition method according to claim 1, characterized in that: The performing environmental adaptability enhancement processing on the initial environmental image to obtain the environmental enhanced image of the target vehicle includes: The initial environment image is corrected using a preset affine transformation matrix to obtain a corrected image of the initial environment image, wherein the preset affine transformation matrix is: Among them, a and d represent the scaling factors along the x-axis and y-axis respectively, b and c represent the shearing factors along the x-axis and y-axis respectively, (t x ,t y ) is the translation vector, A is the affine transformation matrix; The contrast of the corrected image is adjusted, and the brightness of the corrected image after the contrast adjustment is adjusted to obtain an environment enhanced image of the target vehicle.

3. The driverless car visual recognition method according to claim 1, characterized in that: The step of extracting image features of the environment enhanced image comprises: Dividing the environment enhanced image into regions to obtain a region image of the environment enhanced image; Identifying regional objects in the regional image one by one; Marking feature points of objects in the region to obtain key feature points of objects in the region; The image features of the environment enhanced image are generated according to the key feature points.

4. The driverless car visual recognition method according to claim 1, characterized in that: The method of performing feature classification on the image features using a pre-trained convolutional neural network model to obtain classification features of the image features includes: Inputting the image features into a pre-trained convolutional neural network model, and performing multi-layer convolution on the image features using the pre-trained convolutional neural network model to obtain convolution features of the image features; Performing a pooling operation on the convolutional features to obtain pooled features of the convolutional features; A probability distribution of the pooled features is generated, and classification features of the image features are determined according to the probability distribution.

5. The driverless car visual recognition method according to claim 4, characterized in that: The classification features include: variable features and fixed features, wherein the variable features include: pedestrians and vehicles, and the fixed features include: buildings, trees and road signs.

6. The driverless car visual recognition method according to claim 1, characterized in that: The step of performing trajectory prediction on the change feature in the classification feature to obtain the motion trajectory of the change feature includes: Obtaining the initial state of the change feature in the current frame; determining a movement trend of the change characteristic; A motion trajectory of the changing feature is generated according to the initial state and the motion trend.

7. The driverless car visual recognition method according to claim 1, characterized in that: The generating the traffic scene of the target vehicle according to the fixed features in the classification features and the preset energy function comprises: The static environment elements of the current frame are determined according to the fixed features in the classification features and a preset energy function, wherein the preset energy function is: Where E(X) is the energy function, N is the total number of static environmental elements, and X i is the state of the ith static environmental element, X j is the state of the jth static environmental element, h i is the tendency of the i-th static environmental element itself, J ij is the interaction intensity between the ith static environmental element and the jth static environmental element; Performing environmental modeling according to the static environmental elements to obtain an initial environmental model of the target vehicle; The initial environment model is enhanced in detail to obtain the traffic scene of the target vehicle.

8. The driverless car visual recognition method according to any one of claims 1 to 7, characterized in that: The generating the path planning of the target vehicle according to the motion trajectory and the traffic scene comprises: Generate obstacles to be avoided by the target vehicle according to the motion trajectory and the traffic scene; Generating a safe obstacle avoidance path for the target vehicle according to the obstacle to be avoided; Generate a path plan for the target vehicle based on the shortest path in the safe obstacle avoidance path.

9. A visual recognition device for an unmanned vehicle, characterized in that: The device comprises: An image processing module is used to collect an initial environment image of a target vehicle, perform environmental adaptability enhancement processing on the initial environment image, and obtain an environmental enhanced image of the target vehicle; A feature extraction module, used to extract image features of the environment enhanced image; A feature classification module, used to perform feature classification on the image features using a pre-trained convolutional neural network model to obtain classification features of the image features; A trajectory prediction module, used for performing trajectory prediction on the change feature in the classification feature to obtain the motion trajectory of the change feature; The traffic scene generation module is used to generate the traffic scene of the target vehicle according to the fixed features in the classification features and a preset energy function, wherein the preset energy function is: Where E(X) is the energy function, N is the total number of static environmental elements, and X i is the state of the ith static environmental element, X j is the state of the jth static environmental element, h i is the tendency of the i-th static environmental element itself, J ij is the interaction intensity between the ith static environmental element and the jth static environmental element; The path planning module is used to generate the path planning of the target vehicle according to the motion trajectory and the traffic scene, perform instruction conversion on the path planning, obtain the control instruction of the target vehicle, and use the control instruction to drive the target vehicle.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the unmanned vehicle visual recognition method as described in any one of claims 1 to 8 is implemented.

Images