Vehicle intelligent navigation method and device, electronic equipment and storage medium

Abstract

The present disclosure provides a vehicle intelligent navigation method and device, electronic equipment and storage medium. The method comprises: acquiring a video stream collected by a target vehicle during automatic navigation; obtaining a video frame from the video stream; determining a region in the video frame that meets the color of the road traffic sign by color difference; obtaining a road traffic sign region image by screening the determined region using shape features; obtaining a road traffic sign feature vector of the road traffic sign region image; processing the road traffic sign feature vector using a trained road traffic sign classification model, predicting the traffic sign category to which the road traffic sign region image belongs; and sending a navigation instruction matched with the predicted traffic sign category of the road traffic sign region image to control the driving of the target vehicle. The method can accurately predict the traffic sign category of the road traffic sign region.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to a vehicle intelligent navigation method, device, electronic device, and storage medium. Background Technology

[0002] With the rapid development of intelligent vehicles and autonomous driving technology, road traffic sign recognition has become a crucial component of safe driving. Road traffic signs are used to warn, prohibit, restrict, and instruct road users. They are generally displayed as markings on signs along both sides of the road or painted directly on the road surface. However, related technologies suffer from poor accuracy in recognizing road traffic sign areas and predicting traffic sign categories.

[0003] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0004] The purpose of this disclosure is to provide a vehicle intelligent navigation method, device, electronic device, and storage medium, which can accurately predict the traffic sign categories in road traffic sign areas.

[0005] Other features and advantages of this disclosure will become apparent from the following detailed description, or may be learned in part from practice of this disclosure.

[0006] This disclosure provides a vehicle intelligent navigation method, comprising: acquiring a video stream collected by a target vehicle during automatic navigation; obtaining video frames from the video stream; determining regions in the video frames that match the colors of road traffic signs by color differences; obtaining road traffic sign region images by filtering the determined regions using shape features; obtaining road traffic sign feature vectors of the road traffic sign region images; processing the road traffic sign feature vectors using a trained road traffic sign classification model to predict the traffic sign category to which the road traffic sign region images belong; and sending navigation commands matching the predicted traffic sign category to control the driving of the target vehicle.

[0007] In some exemplary embodiments of this disclosure, determining the region in a video frame that matches the color of a road traffic sign by color difference includes: scanning each pixel in the video frame using a template of a predetermined size; replacing the value of the center pixel corresponding to each template with the weighted average pixel value of the pixels in the neighborhood determined by each template to obtain a denoised video frame; converting the color format of the denoised video frame from BGR image format to HSV image format; and determining the set of pixels in the denoised HSV image format video frame that are within a preset color threshold range as the region that matches the color of the road traffic sign.

[0008] In some exemplary embodiments of this disclosure, obtaining a road traffic sign area image by filtering using shape features in a determined area includes: obtaining the minimum bounding rectangle of the area; if the length and width of the minimum bounding rectangle are both greater than a preset length, and the aspect ratio of the minimum bounding rectangle is within a preset ratio range, then the area is determined as a road traffic sign area image.

[0009] In some exemplary embodiments of this disclosure, obtaining the feature vector of a road traffic sign area image includes: determining the gradient magnitude and gradient direction corresponding to each pixel in the road traffic sign area image; obtaining a descriptor of a cell unit based on the gradient magnitude and gradient direction corresponding to each pixel, wherein the cell unit includes a first number of pixels; concatenating the descriptors of the cell units to obtain a descriptor of a block unit, wherein the block unit includes a second number of cell units; and concatenating the descriptors of the block units to obtain the feature vector of the road traffic sign area image.

[0010] In some exemplary embodiments of this disclosure, determining the gradient magnitude and gradient direction corresponding to each pixel in a road traffic sign area image includes: determining the horizontal gradient magnitude and vertical gradient magnitude corresponding to each pixel in the road traffic sign area image; and determining the gradient magnitude and gradient direction corresponding to each pixel based on the horizontal gradient magnitude and vertical gradient magnitude corresponding to each pixel.

[0011] In some exemplary embodiments of this disclosure, the above method further includes: acquiring road traffic sign training images and establishing category labels; converting the road traffic sign training images into grayscale images and scaling the grayscale images to a preset size; obtaining traffic sign training feature vectors of the road traffic sign training images; and training the road traffic sign classification model using the category labels of the road traffic sign training images and their traffic sign training feature vectors as inputs.

[0012] In some exemplary embodiments of this disclosure, obtaining road traffic sign training images includes: acquiring a training video stream captured at multiple locations and under multiple environmental conditions; obtaining training video frames from the training video stream; and cropping the training video frames to obtain road traffic sign training images.

[0013] This disclosure provides a vehicle intelligent navigation device, comprising: a video stream acquisition module for acquiring a video stream collected by a target vehicle during automatic navigation; a video frame acquisition module for obtaining video frames from the video stream; a region determination module for determining regions in the video frames that match the colors of road traffic signs based on color differences; a road traffic sign region image acquisition module for obtaining road traffic sign region images by filtering the determined regions using shape features; a feature vector acquisition module for obtaining road traffic sign feature vectors of the road traffic sign region images; a sign category prediction module for processing the road traffic sign feature vectors using a trained road traffic sign classification model to predict the traffic sign category to which the road traffic sign region images belong; and a command sending module for sending navigation commands matching the predicted traffic sign category to which the road traffic sign region images belong, for controlling the driving of the target vehicle.

[0014] This disclosure provides an electronic device, including: at least one processor; and a storage device for storing at least one program, which, when executed by the at least one processor, causes the at least one processor to implement any of the vehicle intelligent navigation methods described above.

[0015] This disclosure provides a computer-readable storage medium storing a computer program thereon, characterized in that the computer program, when executed by a processor, implements any of the vehicle intelligent navigation methods described above.

[0016] This disclosure provides a vehicle intelligent navigation method that obtains video frames from a video stream captured during the automatic navigation process of a target vehicle. By determining whether the values ​​of each pixel in the video frame are within a color threshold range, the method can accurately identify areas in the video frame that match the colors of road signs. Using shape features, road traffic sign area images can be accurately filtered out from these areas. By obtaining the road traffic sign feature vectors of the road traffic sign area images, the computational load of subsequent models can be reduced, shortening the computation time. Using a trained road traffic sign classification model, the traffic sign category of the road traffic sign area image can be accurately predicted. Based on the predicted traffic sign category, a matching navigation command is sent, accurately controlling the driving of the target vehicle. Furthermore, this method can effectively achieve automatic navigation of the target vehicle during its journey, significantly reducing labor costs, increasing transportation efficiency, reducing safety hazards, and increasing economic benefits.

[0017] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0018] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure. It is obvious that the drawings described below are merely some embodiments of this disclosure, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0019] Figure 1 A schematic diagram of an exemplary system architecture for which the vehicle intelligent navigation method of embodiments of the present disclosure can be applied is shown.

[0020] Figure 2 This is a flowchart illustrating a vehicle intelligent navigation method according to an exemplary embodiment.

[0021] Figure 3 This is a flowchart illustrating another intelligent vehicle navigation method according to an exemplary embodiment.

[0022] Figure 4 This is a flowchart illustrating another intelligent vehicle navigation method according to an exemplary embodiment.

[0023] Figure 5 This is a flowchart illustrating another intelligent vehicle navigation method according to an exemplary embodiment.

[0024] Figure 6 This is a block diagram illustrating a vehicle intelligent navigation device according to an exemplary embodiment.

[0025] Figure 7 A schematic diagram of the structure of an electronic device is shown according to an exemplary embodiment. Detailed Implementation

[0026] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, they are provided so that this disclosure will be more comprehensive and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0027] Furthermore, the accompanying drawings are merely illustrative of this disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0028] Figure 1 A schematic diagram of an exemplary system architecture for which the vehicle intelligent navigation method of embodiments of the present disclosure can be applied is shown.

[0029] like Figure 1 As shown, the system architecture may include server 101, network 102, and terminal device 103. Network 102 serves as the medium for providing a communication link between terminal device 103 and server 101. Network 102 may include various connection types, such as wired or wireless communication links or fiber optic cables, etc.

[0030] Server 101 can be a server that provides various services, such as a backend management server that supports the device operated by the user using terminal device 103. The backend management server can analyze and process received requests and other data, and feed the processing results back to the terminal device.

[0031] The terminal device 103 can be a mobile terminal such as a mobile phone or tablet computer. The terminal device 103 can be placed inside the target vehicle and can control the target vehicle to perform automatic navigation.

[0032] Terminal device 103 can collect video streams during the automatic navigation of the target vehicle and send the collected video streams to server 101; server 101 can receive the video streams collected during the automatic navigation of the target vehicle; server 101 can obtain video frames from the video stream; server 101 can determine the areas in the video frames that match the colors of road traffic signs by using color differences; server 101 can use shape features to filter and obtain road traffic sign area images in the determined areas; server 101 can obtain the road traffic sign feature vectors of the road traffic sign area images; server 101 can use a trained road traffic sign classification model to process the road traffic sign feature vectors and predict the traffic sign category to which the road traffic sign area image belongs; server 101 can send navigation instructions matching the predicted traffic sign category to the terminal device 103; terminal device 103 can accept navigation instructions and control the driving of the target vehicle according to the navigation instructions.

[0033] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Server 101 can be a single physical server, a server cluster consisting of multiple servers, or a cloud server. Depending on actual needs, it can have any number of terminal devices, networks, and servers.

[0034] The steps of the vehicle intelligent navigation method in the exemplary embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings and examples.

[0035] Figure 2 This is a flowchart illustrating a vehicle intelligent navigation method according to an exemplary embodiment. The method provided in this disclosure can be derived from, for example... Figure 1 This disclosure is executed on the server shown, but is not limited thereto.

[0036] like Figure 2 As shown, the vehicle intelligent navigation method provided in this embodiment may include the following steps.

[0037] In step S202, the video stream collected by the target vehicle during the automatic navigation process is acquired.

[0038] In this embodiment of the disclosure, the target vehicle can be a sedan, truck, bus, public transport vehicle, motorcycle, electric vehicle, etc.

[0039] In this embodiment of the disclosure, during automatic navigation, the target vehicle can automatically collect video streams, or the video streams can be automatically collected by a terminal device or camera device placed on the target vehicle; the video streams can be videos that include road information during the journey, and the road information can include road traffic signs.

[0040] In step S204, video frames are obtained from the video stream.

[0041] In this embodiment of the disclosure, each frame of the video stream can be acquired as a video frame and saved in a preset format. The preset format can be JPG (Joint Photographic Experts Group) format. The image size of the video frame can be saved as 2048*1536, and the horizontal and vertical resolutions can be set to 96 DPI (Dots Per Inch).

[0042] In step S206, the area in the video frame that matches the color of the road traffic sign is determined by color difference.

[0043] In this embodiment of the disclosure, multiple video frames can be obtained from a video stream. Taking one video frame as an example, the following description will be provided.

[0044] In this embodiment of the disclosure, the region in the video frame that matches the color of a road sign can be determined by judging whether the value of each pixel in the video frame is within the color threshold range. The region that matches the color of a road traffic sign can be any area where a road traffic sign may exist. For example, the region that matches the color of a road traffic sign can be a blue area. Those skilled in the art can also set other colored areas as needed, and this disclosure does not impose any limitations on this.

[0045] For example, a color threshold range can be preset, and the set of pixels in the video frame that fall within the color threshold range can be used as the area that matches the color of the road traffic sign.

[0046] In step S208, the road traffic sign area image is obtained by filtering the determined area using shape features.

[0047] The road traffic signs may include at least one of the following: Down sign, Forbid sign, Left turn sign, Stop (P) sign, Right turn sign, S-curve sign, U-turn sign, and Up sign; the road sign area image may be an image containing road traffic signs.

[0048] In this embodiment of the disclosure, images of road traffic sign areas can be filtered based on the shape characteristics of the area.

[0049] For example, the minimum bounding rectangle of a region can be obtained, and it can be determined whether the length, width, and aspect ratio of the minimum bounding rectangle of the region meet preset conditions. The region that meets the preset conditions is then used as the road traffic sign region image. The preset conditions can be set as needed, and this disclosure does not impose any restrictions on them.

[0050] In an exemplary embodiment, the minimum bounding rectangle of the region is obtained; if the length and width of the minimum bounding rectangle are both greater than a preset length, and the aspect ratio of the minimum bounding rectangle is within a preset ratio range, then the region is determined as a road traffic sign region image.

[0051] The preset length and preset ratio range can be set according to actual needs, and the preset ratio can be a preset length-to-width ratio.

[0052] For example, the preset length can be 50 pixels, and the preset aspect ratio range can be [0.7, 1.2].

[0053] For example, it can be determined whether the length and width of the minimum bounding rectangle of the region are both greater than 50 pixels, and whether the aspect ratio of the minimum bounding rectangle is within the range of [0.7, 1.2]. If the length and width of the minimum bounding rectangle of the region are both greater than 50 pixels, and the aspect ratio of the minimum bounding rectangle is within the range of [0.7, 1.2], then this region can be cropped and identified as the road traffic sign region image. If the region is tilted, it can be rotated to a horizontal position according to the tilt angle, and the rotated region can be cropped as the road traffic sign region image.

[0054] In step S210, the feature vector of the road traffic sign in the road traffic sign area image is obtained.

[0055] Among them, the feature vector of road traffic signs can be the feature vector of histogram of oriented gradient (HOG).

[0056] In this embodiment of the disclosure, the directional gradient histogram feature vector of the road traffic sign area image can be extracted as the road traffic sign feature vector of the road traffic sign area image.

[0057] For example, the directional gradient histogram algorithm can be used to calculate the gradient magnitude and gradient direction of each pixel in the road traffic sign area image, and the road traffic sign feature vector of the road traffic sign area image can be obtained based on the gradient magnitude and gradient direction of each pixel.

[0058] In step S212, the trained road traffic sign classification model is used to process the feature vectors of road traffic signs and predict the traffic sign category to which the road traffic sign area image belongs.

[0059] The traffic sign categories can include at least one of the following: reverse, no entry, left turn, stop, right turn, curve, U-turn, and forward; the road traffic sign classification model can be an SVM (Support Vector Machine) model.

[0060] In this embodiment of the disclosure, traffic sign categories can be represented by words or decimal numbers. For example, reverse can be represented by 1, forward by 2, left turn by 3, stop by 4, right turn by 5, curve by 6, U-turn by 7, and forward by 8.

[0061] In this embodiment of the disclosure, the feature vector of a road traffic sign can be input into a trained road traffic sign classification model. By processing the feature vector of the road traffic sign using the road traffic sign classification model, the traffic sign category to which the road traffic sign area image belongs can be predicted.

[0062] For example, the SVM model can be used to process the directional gradient histogram feature vector of the road traffic sign area to predict the traffic sign category of the road traffic sign area image, such as left turn.

[0063] In step S214, a navigation command matching the predicted traffic sign category of the road traffic sign area image is sent to control the driving of the target vehicle.

[0064] The navigation instructions sent are matched with the predicted traffic sign category. For example, if the predicted traffic sign category is left turn, then a left turn navigation instruction can be sent to the target vehicle.

[0065] In this embodiment of the present disclosure, a navigation command matching the traffic sign category can be sent to the target vehicle or the terminal device on the target vehicle based on the traffic sign category to which the predicted road traffic sign area image belongs, and the target vehicle can operate according to the navigation command.

[0066] The vehicle intelligent navigation method provided in this disclosure obtains video frames from a video stream collected during the automatic navigation process of a target vehicle. By determining whether the values ​​of each pixel in the video frame are within a color threshold range, the region in the video frame that matches the color of a road sign can be accurately identified. Shape features can be used to accurately filter out road traffic sign area images from these regions. By obtaining the road traffic sign feature vectors of the road traffic sign area images, the computational load of subsequent models can be reduced, shortening the computation time. A trained road traffic sign classification model can accurately predict the traffic sign category to which the road traffic sign area image belongs. Based on the predicted traffic sign category, a matching navigation command is sent, accurately controlling the driving of the target vehicle. Furthermore, this method can effectively achieve automatic navigation of the target vehicle during its journey, significantly reducing labor costs, increasing transportation efficiency, reducing safety hazards, and increasing economic benefits.

[0067] Figure 3 This is a flowchart illustrating another intelligent vehicle navigation method according to an exemplary embodiment.

[0068] In this embodiment of the disclosure, Figure 3 The vehicle intelligent navigation method shown is Figure 2 The vehicle intelligent navigation method shown provides a detailed description of the step of determining the area in the video frame that matches the color of the road traffic sign by color difference, which is also an embodiment of the above step S206.

[0069] like Figure 3 As shown, step S206 may include the following steps.

[0070] In step S302, each pixel in the video frame is scanned using a template of a predetermined size.

[0071] The pre-set size can be set according to actual needs.

[0072] For example, the predetermined size can be 7×7.

[0073] For example, a 7x7 template can be used to scan every pixel in a video frame.

[0074] In step S304, the value of the center pixel of each template is replaced by the weighted average pixel value of the pixels in the neighborhood determined by each template to obtain the denoised video frame.

[0075] In this context, the neighborhood of each template consists of pixels excluding the center pixel within the template. For example, if the template size is 7*7, then the neighborhood of the template is a 7*7-1 pixel area excluding the center pixel.

[0076] In this embodiment, a Gaussian denoising method can be used to replace the value of the center pixel corresponding to each template with the weighted average pixel value of the pixels in the neighborhood determined by each template, resulting in a denoised video frame. This method can smooth the image and reduce the impact of noise in the image on color conversion analysis.

[0077] In step S306, the color format of the denoised video frame is converted from BGR image format to HSV image format.

[0078] In this embodiment of the disclosure, the color format of the denoised video frame can be converted from BGR (Blue Green Red) image format to HSV (Hue Saturation Value) image format.

[0079] Among them, HSV type images can more intuitively reflect the color information of an image.

[0080] In step S308, the set of pixels in the denoised HSV image format video frame that are within the preset color threshold range is determined as the area that matches the color of the road traffic sign.

[0081] The preset color threshold range can be set according to actual needs.

[0082] For example, the highest value of the preset color threshold range can be [124, 255, 255], the lowest value of the preset color threshold range can be [60, 100, 60], and the area that matches the color of the road traffic sign can be the blue area.

[0083] For example, the set of pixels in the denoised video frame of HSV image format that are between [60, 100, 60] and [124, 255, 255] can be regarded as the area that matches the color of the road traffic sign, and the set of pixels that are not between [60, 100, 60] and [124, 255, 255] can be regarded as the interference area.

[0084] In this embodiment of the disclosure, in order to reduce the impact of changes in shooting angle on the recognition accuracy of the subsequent road traffic sign classification model, regions that match the color of the road traffic sign can be determined, regions that match the shape features can be selected from the determined regions, the selected regions can be rotated to the horizontal according to the tilt angle, and then the rotated regions can be cropped as the road traffic sign image.

[0085] Figure 4 This is a flowchart illustrating another intelligent vehicle navigation method according to an exemplary embodiment.

[0086] In this embodiment of the disclosure, Figure 4 The vehicle intelligent navigation method shown is Figure 2 The vehicle intelligent navigation method shown provides a detailed description of the steps for obtaining the feature vector of the road traffic sign area image, that is, it provides an embodiment of the above step S210.

[0087] like Figure 4 As shown, step S210 may include the following steps.

[0088] In step S402, the gradient magnitude and gradient direction of each pixel in the road traffic sign area image are determined.

[0089] In this embodiment of the disclosure, the HOG (Histogram of Oriented Gradient) algorithm can be used to calculate the gradient magnitude and gradient direction of each pixel in the road traffic sign area image.

[0090] The HOG algorithm operates at the cellular level of the image, thus reducing the influence of geometric and optical conditions. For example, under conditions of coarse spatial sampling, fine directional sampling, and strong local optical normalization, the features extracted by this algorithm have a negligible impact on the recognition results when the magnitude of image skew is small. Furthermore, since the HOG algorithm lacks rotation and scale invariance, it has low computational cost, significantly reducing computation time in vehicle autonomous navigation.

[0091] The parameters for the directional gradient histogram algorithm can be set as follows: number of unsigned directional partitions: orientation = 9; number of pixels per cell: pixels_per_cell = (8, 8); number of cells per block: cell_per_block = (2, 2); use of Norm type within the block: block_norm = norm; Gamma correction: transform_sqrt = True.

[0092] In an exemplary embodiment, the horizontal gradient magnitude and vertical gradient magnitude corresponding to each pixel in the road traffic sign area image can be determined.

[0093] For example, the horizontal and vertical gradient magnitudes for each pixel can be determined using the following formula:

[0094] G x (x,y)=H(x+1,y)-H(x-1,y)

[0095] G y (x,y)=H(x,y+1)-H(x,y-1)

[0096] Among them, G x (x,y) represents the horizontal gradient magnitude at pixel (x,y), G y (x,y) represents the vertical gradient magnitude at pixel (x,y), and H(x,y) represents the pixel value at pixel (x,y).

[0097] In an exemplary embodiment, the gradient magnitude and gradient direction of each pixel can be determined based on the horizontal and vertical gradient magnitudes of each pixel.

[0098] For example, the gradient magnitude and gradient direction for each pixel can be determined using the following formula:

[0099]

[0100]

[0101] Where G(x,y) represents the gradient magnitude of pixel (x,y), and α(x,y) represents the gradient direction of pixel (x,y).

[0102] In this embodiment of the disclosure, determining the gradient magnitude and gradient direction of each pixel in the road traffic sign area image can capture contour information, brighten darker areas, and reduce the impact of shadows and lighting changes on the image.

[0103] In step S404, a descriptor for a cell unit is obtained based on the gradient magnitude and gradient direction corresponding to each pixel. The cell unit includes a first number of pixels.

[0104] The first quantity can be set as needed. For example, the first quantity can be 8×8, that is, one cell unit can include 8×8 pixels.

[0105] In this embodiment of the disclosure, the gradient direction can be divided into 9 direction blocks in 360°. Each pixel in the cell unit is weighted onto the corresponding direction block according to its corresponding gradient direction projection and gradient magnitude, so that the gradient direction histogram of the cell unit can be obtained, and a 9-dimensional feature vector can be obtained. The 9-dimensional feature vector can be used as the descriptor of the cell unit.

[0106] In step S406, the descriptors of the cell units are concatenated to obtain the descriptors of the block units, and the block units include a second number of cell units.

[0107] The second quantity can be set as needed. For example, the second quantity can be 2×2, that is, one block unit can include 2×2 cell units.

[0108] In this embodiment of the disclosure, the descriptors of each cell unit in the block unit can be concatenated to obtain the descriptor of the block unit.

[0109] In step S408, the descriptors of the block cells are concatenated to obtain the road traffic sign feature vector of the road traffic sign area image.

[0110] In this embodiment of the disclosure, the descriptors of each block unit in the road traffic sign area image can be concatenated to obtain the road traffic sign feature vector of the road traffic sign area image, which is the representation of the road traffic sign area image.

[0111] Figure 5 This is a flowchart illustrating another intelligent vehicle navigation method according to an exemplary embodiment.

[0112] In this embodiment of the disclosure, Figure 2 Based on the vehicle intelligent navigation method shown, Figure 5 The vehicle intelligent navigation method shown may also include the following steps.

[0113] In step S502, training images of road traffic signs are acquired and category labels are established.

[0114] In an exemplary embodiment, a training video stream captured at multiple locations and under multiple environmental conditions is acquired; training video frames are obtained from the training video stream; and the training video frames are cropped to obtain road traffic sign training images.

[0115] In this embodiment, within an intelligent rail transit system, cameras can capture images of each type of road traffic sign from different locations and under varying lighting conditions to obtain a training video stream. The focus and object distance are adjusted until the traffic sign is visible before acquisition begins. After the traffic sign training video stream is acquired, each frame can be obtained and saved as a JPG (image size 2048*1536, horizontal and vertical resolution both 96 DPI) as a training video frame. The training video frames can be cropped to obtain road traffic sign training images for each frame. These images are then categorized and labeled. A training set of 2000 images from each category can be selected to construct the road traffic sign classification model.

[0116] Category labels can be marked with decimal numbers.

[0117] In step S504, the road traffic sign training image is converted into a grayscale image, and the grayscale image is scaled to a preset size.

[0118] In this embodiment, before performing feature analysis and extraction on the road traffic sign training image, the road traffic sign training image can be converted into a grayscale image. The grayscale value ranges from 0 to 255, meaning the original three-dimensional image is transformed into a one-dimensional image between 0 and 255 according to image type conversion rules. 0 represents black, the lowest brightness value, and 255 represents white, the highest brightness value; intermediate values ​​represent different brightness levels of gray between black and white. The grayscale image can clearly show the contours of different objects in the image based on brightness, facilitating the analysis and extraction of the entire feature information of the road traffic sign training image.

[0119] In this embodiment of the disclosure, the grayscale image can be scaled to a preset size to accelerate feature analysis and extraction. The preset size can be set according to actual conditions; for example, the preset size can be 64×64.

[0120] In step S506, the traffic sign training feature vector of the road traffic sign training image is obtained.

[0121] In this embodiment of the disclosure, step S506 can refer to step S210 described above, and will not be repeated here.

[0122] In step S508, the category labels of the road traffic sign training images and their traffic sign training feature vectors are used as input to train the road traffic sign classification model.

[0123] In this embodiment, the road traffic sign classification model can be an SVM model. Because SVM models largely do not involve probability measures and the law of large numbers, they can simplify common classification and regression problems. The computational complexity of an SVM model depends on the number of support vectors, rather than the dimensionality of the sample space, thus avoiding the "curse of dimensionality" to some extent. Furthermore, SVM models can not only capture key samples but also eliminate redundancy, exhibiting excellent robustness.

[0124] In this embodiment, the "one-against-one" method from the SVM model can be used. That is, n*(n-1) / 2 classifiers can be reconstructed according to the required number of classifications n, and each classifier is trained on two types of road traffic sign training images. Training an SVM classifier is equivalent to solving a quadratic programming problem.

[0125] In this embodiment of the disclosure, the training parameters of the SVM model can be set as follows: penalty parameter C = 10; kernel function is set to Gaussian, i.e., kernel = rbf; the dimension degree of the polynomial function is set to 3; the error value for stopping training is set to tol = 1e-5; and probability estimation is used with probability = True.

[0126] Each SVM classifier can be trained on any two classes of road traffic sign training images (without repetition), until all classifiers have trained on all road traffic sign training images. Assuming the training image is of class i and class j, the training principle can be:

[0127]

[0128]

[0129]

[0130]

[0131] Where w represents the hyperplane normal vector, and w is the unique optimization objective; b represents the hyperplane intercept; ε represents the slack variable; and the subscript t represents the index of the sample in the union of the i-th and j-th classes of data. It represents a nonlinear mapping from the input space to the feature space.

[0132] In this embodiment of the disclosure, after the SVM model is trained, the input road traffic sign feature vector can be processed according to the trained SVM model to predict the traffic sign category of the road traffic sign area.

[0133] In this embodiment of the disclosure, each trained SVM model can be configured according to the decision function on the input road traffic sign feature vector X. new There is a prediction (vote). Taking the SVM model prediction between class i and class j as an example, if X... new If the prediction is for class i, then class i receives 1 vote; otherwise, class j receives 1 vote. The class with the most votes is the model's prediction of the feature vector of the road traffic sign. The decision function is the basis for the SVM model to judge the input data, and it is as follows:

[0134]

[0135] Among them, X new This represents the input data that needs to be predicted; This represents the predicted result. In this embodiment of the disclosure, X new This represents the input feature vector of road traffic signs. Indicates the predicted traffic sign category.

[0136] It should be noted that the above figures are merely illustrative of the processes included in the method according to exemplary embodiments of this disclosure, and are not intended to be limiting. It is readily understood that the processes shown in the above figures do not indicate or limit the temporal order of these processes. Furthermore, it is readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0137] The following are embodiments of the apparatus disclosed herein, which can be used to execute embodiments of the method disclosed herein. For details not disclosed in the apparatus embodiments of this disclosure, please refer to the embodiments of the method disclosed herein.

[0138] Figure 6 This is a block diagram illustrating a vehicle intelligent navigation device according to an exemplary embodiment.

[0139] like Figure 6 As shown, the vehicle intelligent navigation device 600 may include: a video stream acquisition module 602, a video frame acquisition module 604, a region determination module 606, a road traffic sign region image acquisition module 608, a feature vector acquisition module 610, a sign category prediction module 612, and a command sending module 614.

[0140] The system includes the following modules: a video stream acquisition module 602, which acquires video streams collected by the target vehicle during automatic navigation; a video frame acquisition module 604, which obtains video frames from the video stream; a region determination module 606, which determines regions in the video frames that match the colors of road traffic signs based on color differences; a region image acquisition module 608, which uses shape features to filter and obtain road traffic sign region images within the determined regions; a feature vector acquisition module 610, which obtains road traffic sign feature vectors from the road traffic sign region images; a sign category prediction module 612, which uses a trained road traffic sign classification model to process the road traffic sign feature vectors and predict the traffic sign category to which the road traffic sign region images belong; and a command sending module 614, which sends matching navigation commands based on the predicted traffic sign category to which the road traffic sign region images belong, to control the driving of the target vehicle.

[0141] In an exemplary embodiment, the region determination module 606 may include: a template obtaining unit, which can be used to scan each pixel in the video frame using a template of a predetermined size; a denoising unit, which can be used to replace the value of the center pixel corresponding to each template with the weighted average pixel value of the pixels in the neighborhood determined by each template, so as to obtain a denoised video frame; a format conversion unit, which can be used to convert the color format of the denoised video frame from BGR image format to HSV image format; and a region determination unit, which can be used to determine the set of pixels in the denoised HSV image format video frame within a preset color range as the region that conforms to the color of the road traffic sign.

[0142] In an exemplary embodiment, the region image acquisition module 608 may include: a bounding rectangle acquisition unit, which can be used to acquire the minimum bounding rectangle of the region; and a region image acquisition unit, which can be used to determine the region as a road traffic sign region image if the length and width of the minimum bounding rectangle are both greater than a preset length and the aspect ratio of the minimum bounding rectangle is within a preset ratio range.

[0143] In an exemplary embodiment, the feature vector acquisition module 610 may include: a gradient determination unit, which can be used to determine the gradient magnitude and gradient direction corresponding to each pixel in the road traffic sign area image; a descriptor acquisition unit, which can be used to obtain a descriptor of a cell unit based on the gradient magnitude and gradient direction corresponding to each pixel, wherein the cell unit includes a first number of pixels; a first concatenation unit, which can be used to concatenate the descriptors of the cell units to obtain a descriptor of a block unit, wherein the block unit includes a second number of cell units; and a second concatenation unit, which can be used to concatenate the descriptors of the block units to obtain a road traffic sign feature vector of the road traffic sign area image.

[0144] In an exemplary embodiment, the gradient determination unit may include: a gradient magnitude determination unit, which can be used to determine the horizontal gradient magnitude and vertical gradient magnitude corresponding to each pixel in the road traffic sign area image; and a magnitude and direction determination unit, which can be used to determine the gradient magnitude and gradient direction corresponding to each pixel based on the horizontal gradient magnitude and vertical gradient magnitude corresponding to each pixel.

[0145] In an exemplary embodiment, the vehicle intelligent navigation device 600 may further include: a training image acquisition module, which can be used to acquire road traffic sign training images and establish category labels; an image conversion module, which can be used to convert the road traffic sign training images into grayscale images and scale the grayscale images to a preset size; a training feature vector acquisition module, which can be used to obtain traffic sign training feature vectors of the road traffic sign training images; and a model training module, which can be used to train a road traffic sign classification model by using the category labels of the road traffic sign training images and their traffic sign training feature vectors as input.

[0146] In an exemplary embodiment, the training image acquisition module may include: a training video stream acquisition unit, which can be used to acquire training video streams captured at multiple locations and under multiple environmental conditions; a training video frame acquisition unit, which can be used to obtain training video frames from the training video stream; and a training video frame cropping unit, which can be used to crop the training video frames to obtain road traffic sign training images.

[0147] It should be noted that the block diagrams shown in the above figures are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0148] Figure 7 This is a schematic diagram illustrating the structure of an electronic device according to an exemplary embodiment. It should be noted that... Figure 7 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments of the present invention.

[0149] like Figure 7As shown, the electronic device 700 includes a central processing unit (CPU) 701, which can perform various appropriate actions and processes based on a program stored in a read-only memory (ROM) 702 or a program loaded from a storage section 708 into a random access memory (RAM) 703. The RAM 703 also stores various programs and data required for the operation of the system 700. The CPU 701, ROM 702, and RAM 703 are interconnected via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0150] The following components are connected to the I / O interface 705: an input section 706 including a keyboard, mouse, etc.; an output section 707 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 708 including a hard disk, etc.; and a communication section 709 including a network interface card such as a LAN card, modem, etc. The communication section 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the I / O interface 705 as needed. A removable medium 711, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 710 as needed so that computer programs read from it can be installed into the storage section 708 as needed.

[0151] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 709, and / or installed from removable medium 711. When the computer program is executed by central processing unit (CPU) 701, it performs the functions defined above in the system of the present invention.

[0152] It should be noted that the computer-readable medium shown in this invention can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this invention, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0153] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0154] The units described in the embodiments of the present invention can be implemented in software or hardware. The described units can also be housed in a processor; for example, a processor can be described as including a sending unit, an acquisition unit, a determining unit, and a first processing unit. The names of these units do not necessarily limit the specific unit; for example, the sending unit can also be described as "a unit that sends an image acquisition request to a connected server."

[0155] In another aspect, the present invention also provides a computer-readable medium, which may be included in the device described in the above embodiments; or it may exist independently and not assembled into the device. The computer-readable medium carries one or more programs that, when executed by the device, cause the device to include: acquiring a video stream collected by a target vehicle during automatic navigation; obtaining video frames from the video stream; determining regions in the video frames that match the colors of road traffic signs based on color differences; filtering the determined regions using shape features to obtain road traffic sign region images; obtaining road traffic sign feature vectors from the road traffic sign region images; processing the road traffic sign feature vectors using a trained road traffic sign classification model to predict the traffic sign category to which the road traffic sign region images belong; and sending navigation commands matching the predicted traffic sign category to control the driving of the target vehicle.

[0156] Exemplary embodiments of the present invention have been specifically shown and described above. It should be understood that the present invention is not limited to the detailed structures, arrangements, or implementation methods described herein; rather, the present invention is intended to cover various modifications and equivalent arrangements contained within the spirit and scope of the appended claims.

Claims

1. A method for intelligent navigation of a vehicle, the method comprising: The method comprises: acquiring a video stream collected by a target vehicle during automatic navigation; obtaining a video frame from the video stream; determining a region in the video frame that matches a road traffic sign color through color difference; screening a road traffic sign region image in the determined region through shape features; obtaining a road traffic sign feature vector of the road traffic sign region image; processing the road traffic sign feature vector through a trained road traffic sign classification model to predict a traffic sign category to which the road traffic sign region image belongs; sending a navigation instruction matched with the predicted traffic sign category to control the driving of the target vehicle; wherein the screening of the road traffic sign region image in the determined region through shape features comprises: acquiring a minimum bounding rectangle of the region; if the length and width of the minimum bounding rectangle are both greater than a preset length, and the length-width ratio of the minimum bounding rectangle is within a preset ratio range, rotating the region to be horizontal according to the inclination angle of the region, and then intercepting the rotated region as a road traffic sign region image; the determination of the region in the video frame that matches a road traffic sign color through color difference comprises: scanning each pixel in the video frame through a template of a predetermined size; replacing the value of the center pixel point corresponding to each template with the weighted average pixel value of the pixels in the neighborhood determined by each template to obtain a denoised video frame; converting the color format of the denoised video frame from BGR image format to HSV image format; determining a set of pixel points in the denoised video frame in HSV image format within a preset color threshold range as the region that matches the road traffic sign color.

2. The method of claim 1, wherein, the obtaining of the road traffic sign feature vector of the road traffic sign region image comprises: determining the gradient amplitude and gradient direction corresponding to each pixel point in the road traffic sign region image; obtaining a cell unit descriptor according to the gradient amplitude and gradient direction corresponding to each pixel point, the cell unit comprising a first number of pixel points; concatenating the cell unit descriptors to obtain a block unit descriptor, the block unit comprising a second number of cell units; concatenating the block unit descriptors to obtain the road traffic sign feature vector of the road traffic sign region image.

3. The method of claim 2, wherein, the determination of the gradient amplitude and gradient direction corresponding to each pixel point in the road traffic sign region image comprises: determining the horizontal direction gradient amplitude and vertical direction gradient amplitude corresponding to each pixel point in the road traffic sign region image; determining the gradient amplitude and gradient direction corresponding to each pixel point according to the horizontal direction gradient amplitude and vertical direction gradient amplitude corresponding to each pixel point.

4. The method of claim 1, wherein, The method further comprises: acquiring road traffic sign training images and setting category labels; converting the road traffic sign training images into grayscale images and scaling the grayscale images to a preset size; obtaining traffic sign training feature vectors of the road traffic sign training images; The road traffic sign classification model is trained by taking the category label of the road traffic sign training image and the traffic sign training feature vector thereof as input.

5. The method of claim 4, wherein, The road traffic sign training image is obtained by: Obtaining a training video stream captured at multiple locations and under multiple environmental conditions; Obtaining training video frames from the training video stream; Obtaining the road traffic sign training image by cropping the training video frames.

6. A vehicle intelligent navigation device, characterized by, Comprise: A video stream acquisition module configured to acquire a video stream collected by a target vehicle during automatic navigation; A video frame obtaining module configured to obtain video frames from the video stream; A region determining module configured to determine a region in the video frames that meets the color of road traffic signs by color difference; A road traffic sign region image obtaining module configured to obtain a road traffic sign region image by shape feature screening in the determined region; A feature vector obtaining module configured to obtain a road traffic sign feature vector of the road traffic sign region image; A sign category predicting module configured to process the road traffic sign feature vector by using a trained road traffic sign classification model to predict the traffic sign category to which the road traffic sign region image belongs; An instruction sending module configured to send a navigation instruction matched with the predicted traffic sign category of the road traffic sign region image for controlling the driving of the target vehicle; The road traffic sign region image obtaining module is configured to: obtain a minimum bounding rectangle of the region; if the length and width of the minimum bounding rectangle are both greater than a preset length, and the aspect ratio of the minimum bounding rectangle is within a preset ratio range, then the region is determined as the road traffic sign region image; The region determining module is configured to: scan each pixel in the video frames by using a template of a predetermined size; replace the value of the center pixel point corresponding to each template with the weighted average pixel value of the pixels in the neighborhood determined by each template to obtain a denoised video frame; convert the color format of the denoised video frame from BGR image format to HSV image format; determine the set of pixel points in the denoised video frame in HSV image format that are within a preset color threshold range as the region that meets the color of road traffic signs.

7. An electronic device, comprising: Comprise: At least one processor; A storage device configured to store at least one program, which, when executed by the at least one processor, causes the at least one processor to implement the method of any one of claims 1 to 5.

8. A computer-readable storage medium having stored thereon computer- executable instructions, wherein, The executable instructions, when executed by the processor, implement the method of any one of claims 1 to 5.

Images