Bag-of-word model construction method and device, equipment, storage medium and program product
By introducing dot-line synchronization processing and line feature extraction into the traditional bag of words model, the distance dictionary and line word bag vectors are constructed, which solves the performance degradation of traditional bags in scenarios with large environmental changes and poor texture, and improves the vehicle in the garage Positioning the success rate of initialization.
Patent Information
- Application Number
- CN202411492240.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-24
- Publication Date
- 2025-02-14
AI Technical Summary
The construction of traditional word bags is trained by feature points, which is not very friendly to different scenes with large environmental changes and scenes with poor textures. There are often problems of insufficient features and reduced performance of word bags.
By extracting point features and line features of the perceived images collected by the vehicle during parking, synchronous processing of dots and lines, building distance dictionary and line bag-of-word vectors, and finally building the word bag-of-word model of the vehicle.
This method can better describe the horizontal and vertical structured scenario of the garage, improve the quality of the word bag in the structured scenario, and thus improve the success rate of the vehicle's positioning and initialization in the garage.
Smart Images
Figure CN119445144A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of image recognition technology, and in particular to a bag-of-words model construction method, device, equipment, storage medium and program product. Background Art
[0002] When the memory parking is in an underground garage or in a scenario without GPS signal, when the memory path is completed and the cruise initialization begins, in order to initialize any point, it is necessary to quickly find the position of the path. When there is a GPS signal, GPS rough positioning will be used, but when there is no GPS, only the perception image is available. Usually, the perception image is used to train the word bag vector for preliminary positioning. Among them, traditional visual feature extraction can perform feature tracking, composition, and closed-loop detection in real time, complete synchronous positioning and map construction, and construct a word bag to quickly locate the memory path.
[0003] Traditional visual feature extraction mainly extracts feature points, and traditional bag-of-words construction uses feature points to train bag-of-words. However, point features are easily disturbed by light and noise, and the constructed three-dimensional point map is relatively sparse, making it difficult to express the real scene structure. As a result, the feature point method is not very friendly to different scenes with large environmental changes and scenes with poor textures, and often results in insufficient features and reduced bag-of-words performance. Summary of the invention
[0004] The main purpose of the present application is to provide a method, device, equipment, storage medium and program product for constructing a bag-of-words model, aiming to solve the technical problem that traditional bag-of-words construction uses feature points to train the bag-of-words, which is not very friendly to different scenes with large environmental changes and scenes with poor textures, and often results in insufficient features and reduced bag-of-words performance.
[0005] To achieve the above purpose, the present application proposes a method for constructing a bag-of-words model, the method comprising:
[0006] Extract features from the perception images collected during the parking process to obtain point features and line features;
[0007] Synchronously processing the point feature and the line feature to obtain a synchronized line feature;
[0008] Keyword construction is performed according to the feature point descriptor corresponding to the point feature and the synchronized line feature to obtain a distance dictionary and a line bag-of-words vector;
[0009] A bag-of-words model corresponding to the vehicle is constructed according to the distance dictionary and the line bag-of-words vector.
[0010] In one embodiment, the step of extracting features from the perception image collected during the parking process to obtain point features and line features includes:
[0011] Acquire a perception image collected by the vehicle during the parking process;
[0012] Extracting feature points of the perceived image through a preset deep learning framework to obtain point features of the perceived image;
[0013] Feature lines are extracted from the perceived image using a preset semantic segmentation model to obtain line features of the perceived image.
[0014] In one embodiment, the step of synchronously processing the point feature and the line feature to obtain the synchronized line feature includes:
[0015] Determine the point coordinates of the point feature and the line coordinates of the line feature;
[0016] Perform distance calculation according to the point coordinates and the line coordinates to obtain a distance value;
[0017] When the distance value is less than a preset distance threshold and the point coordinates and the line coordinates meet preset coordinate conditions, the point feature is projected onto the corresponding line feature to obtain a synchronized line feature.
[0018] In one embodiment, the step of constructing keywords based on the feature point descriptors corresponding to the point features and the synchronized line features to obtain a distance dictionary and a line bag-of-words vector includes:
[0019] Determine a feature point descriptor of the point feature, wherein the point feature includes a key point and a feature point descriptor within a preset range of the key point;
[0020] Clustering the feature point descriptors and the synchronized line features to obtain a plurality of distance data;
[0021] constructing a distance dictionary according to the plurality of distance data;
[0022] A line bag-of-words vector is constructed according to the center point and line coordinates of the synchronized line features.
[0023] In one embodiment, the step of constructing a bag-of-words model corresponding to the vehicle according to the distance dictionary and the line bag-of-words vector includes:
[0024] Sorting the line bag-of-words vectors by their identifiers to obtain sorted line bag-of-words vectors;
[0025] When the vehicle is performing simultaneous positioning and map building, recording the position information corresponding to the line bag-of-words vector;
[0026] A bag-of-words model corresponding to the vehicle is constructed according to the distance dictionary, the line bag-of-words vector and the posture information.
[0027] In one embodiment, after the step of constructing the bag-of-words model corresponding to the vehicle according to the distance dictionary and the line bag-of-words vector, the method further includes:
[0028] When receiving a parking instruction, searching the current frame captured by the vehicle camera according to the bag-of-words model to obtain a line bag-of-words vector closest to the current frame;
[0029] Retrieving corresponding posture information according to the identifier of the line bag-of-words vector;
[0030] The position information is used as the initial positioning of the vehicle for parking.
[0031] In addition, to achieve the above purpose, the present application also proposes a bag-of-words model construction device, the device comprising:
[0032] A feature extraction module is used to extract features from the perception images collected during the parking process of the vehicle to obtain point features and line features;
[0033] A point-line synchronization module, used for synchronizing the point features and the line features to obtain synchronized line features;
[0034] A keyword construction module, used to construct keywords according to the feature point descriptors corresponding to the point features and the synchronized line features, to obtain a distance dictionary and a line bag-of-words vector;
[0035] The bag-of-words model module is used to construct a bag-of-words model corresponding to the vehicle according to the distance dictionary and the line bag-of-words vector.
[0036] In addition, to achieve the above-mentioned purpose, the present application also proposes a word bag model construction device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, and the computer program is configured to implement the steps of the word bag model construction method as described above.
[0037] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the word bag model construction method described above are implemented.
[0038] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the steps of the word bag model construction method described above are implemented.
[0039] One or more technical solutions proposed in this application have at least the following technical effects: this application first extracts features from the perceived images collected by the vehicle during parking to obtain point features and line features; then synchronizes the point features and the line features to obtain synchronized line features; then constructs keywords based on the feature point descriptors corresponding to the point features and the synchronized line features to obtain a distance dictionary and a line bag-of-words vector; finally, constructs a bag-of-words model corresponding to the vehicle based on the distance dictionary and the line bag-of-words vector. Since this application obtains point features and line features when extracting features from the perceived images, and performs point-line synchronization to obtain synchronized line features to construct a bag-of-words model, it can better describe horizontally and vertically structured scenes such as garages, improve the quality of the bag-of-words in structured scenes such as garages, and thus improve the success rate of vehicle positioning initialization in garages. BRIEF DESCRIPTION OF THE DRAWINGS
[0040] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.
[0041] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.
[0042] Figure 1 A flowchart of the first embodiment of the bag-of-words model construction method of the present application is provided;
[0043] Figure 2 A flowchart of the word bag construction process provided in the embodiment of the present application;
[0044] Figure 3 A flowchart of the second embodiment of the bag-of-words model construction method of the present application is provided;
[0045] Figure 4 This is a schematic diagram of the module structure of the bag-of-words model construction device of the embodiment of the present application;
[0046] Figure 5 Schematic diagram of the device structure of the hardware operating environment involved in the method for constructing a bag-of-words model in an embodiment of the present application.
[0047] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION
[0048] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.
[0049] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.
[0050] It should be noted that the execution subject of this embodiment can be a computing service device with the functions of feature extraction, point-line synchronization and word bag construction, such as a personal computer, a car computer, etc., or an electronic device capable of realizing the above functions, a word bag model construction device (referred to as a construction device) that executes the word bag model construction method of this application, etc., and this embodiment does not limit this. The following takes the construction device as an example to illustrate this embodiment and the following embodiments.
[0051] Based on this, the present application embodiment provides a method for constructing a bag-of-words model. Figure 1 , Figure 1 A flowchart of the first embodiment of the bag-of-words model construction method of the present application is provided.
[0052] In this embodiment, the bag-of-words model construction method includes steps S10 to S40:
[0053] Step S10: extracting features from the perception images collected during the parking process of the vehicle to obtain point features and line features.
[0054] It should be noted that the perceived image is image information obtained by photographing and capturing the vehicle's surrounding environment during parking operations.
[0055] Exemplarily, the perception image may include road conditions around the vehicle, the location and markings of parking spaces, surrounding obstacles (such as other vehicles, walls, pillars, etc.), etc., to assist the driver in precise parking.
[0056] It should be noted that point features refer to some isolated points with significant characteristics in the perceived image, such as corner points and key points in the perceived image.
[0057] The extraction of point features can be achieved through a visual feature algorithm, such as an ORB algorithm, a SIFT (Scale-Invariant Feature Transform) algorithm, and a Super Point deep learning algorithm, etc., which is not limited in this embodiment.
[0058] It should be noted that the line feature is a line segment or a line with a certain direction extracted from the perceived image, such as the edge and contour of a garage column. The extraction of line features can be achieved by semantic segmentation, which is not limited in this embodiment.
[0059] In this implementation, by extracting point features and line features of the perceived image, key information can be effectively obtained from the image, thereby improving the quality of the bag-of-words model.
[0060] In a feasible implementation, step S10 of this embodiment may include the steps of: acquiring a perceived image collected by the vehicle during parking; extracting feature points of the perceived image through a preset deep learning framework to obtain point features of the perceived image; and extracting feature lines of the perceived image through a preset semantic segmentation model to obtain line features of the perceived image.
[0061] It should be noted that the deep learning framework is a learning framework used for key point detection and description, such as superpoint, TensorFlow and other frameworks, which is not limited in this embodiment.
[0062] The deep learning framework can detect representative key points in the perceived image, generate unique description information for these key points, and obtain the point features of the perceived image.
[0063] It should be noted that the semantic segmentation model is a deep learning model for image semantic segmentation, such as a segnet model, an FCN (fully convolutional network) model, etc., which is not limited in this embodiment.
[0064] The features of the perceived image are extracted through a series of convolutional layers of the semantic segmentation model to obtain feature maps at different levels. Then, the category of each pixel in the feature map is predicted, semantic segmentation is performed, and its category is labeled to obtain the line features of the perceived image.
[0065] In this embodiment, during the feature extraction process, the construction device can use the deep learning framework superpoint to extract feature points and the semantic segmentation model segnet to extract line features, so as to effectively obtain key information from the image.
[0066] Step S20: Synchronize the point features and the line features to obtain synchronized line features.
[0067] It should be noted that, considering that most of the extracted point features are noise points and it is difficult to form effective information, the point features and line features can be synchronized to ensure that the extracted points and lines are not noise points, so that more features are on the line surface to better describe the horizontal and vertical structured scenes such as garages.
[0068] Specifically, the process of point-line synchronization processing may include: firstly, taking the line feature as the center, removing the point features outside a certain range of the line feature to filter out excessive noise; and then gathering the remaining point features and line features to form a new line feature.
[0069] In this embodiment, by performing point-line synchronous processing on point features and line features, it can be ensured that the extracted points and lines are not noise points, so that more features are on the line surface.
[0070] In a feasible implementation, step S20 of this embodiment may include the steps of: determining the point coordinates of the point feature and the line coordinates of the line feature; performing distance calculation based on the point coordinates and the line coordinates to obtain a distance value; when the distance value is less than a preset distance threshold and the point coordinates and the line coordinates meet the preset coordinate conditions, projecting the point feature onto the corresponding line feature to obtain a synchronized line feature.
[0071] It should be noted that the point coordinate point is a set of values used to uniquely determine the position of a feature point in the perceived image plane. For example, in a two-dimensional plane, (x, y) is used to represent the horizontal and vertical coordinates of a point.
[0072] Line coordinates are a set of coordinates of a series of points that make up a line feature. The shape and position of a line can be described by the coordinate sequence of the points in the line coordinates. It can also be represented by the starting point coordinates and the end point coordinates in the line coordinates, which is not limited in this embodiment.
[0073] It should be noted that the distance value is the shortest connection distance between each point feature and the two ends of the line feature. The preset distance threshold is a pre-set value used to determine whether the distance between a point feature and a line feature is within the corresponding line feature. The preset coordinate condition is a pre-set condition used to determine whether a point feature can be projected onto the corresponding line feature.
[0074] Exemplarily, for ease of understanding, assume that the point coordinates of the extracted point feature are Pi (xi, yi), and the extracted line features (such as the edge of the ground, the lines of the pillars, etc.) are represented by a polynomial Li (ai, bi, ci, si, ei), where ai, bi, ci are polynomial coefficients; si, ei are the starting point coordinates and end point coordinates of the line.
[0075] The distance value can now be calculated:
[0076]
[0077] If the distance value is less than a set threshold (the threshold can be obtained through actual testing and is not limited in this embodiment), and xi is before the x value of si and ei, and yi is before the y value of si and ei, then it is considered that the Pi point can be projected onto the Li line; at this time, sufficient Pi points can be used to represent the Li line, thereby obtaining the synchronized line features.
[0078] In this implementation, the distance value is calculated by using the point coordinates and the line coordinates, so that the point feature can be accurately projected onto the corresponding line feature.
[0079] Furthermore, if Figure 2 As shown, Figure 2 A flowchart of the word bag construction process provided in the embodiment of the present application. After extracting feature points and feature lines from the camera image (i.e., the perceived image) and performing point and line synchronization, feature point matching and feature line matching can also be performed to improve the robustness of feature extraction. The feature line matching steps are as follows:
[0080] Assume two thresholds, one is the number of feature points Nij matched between two frames of the perceived image, and the other is the confidence Cij of the number of matched feature points; assuming that the number of features matched between two frames each time is N, then Cij is calculated as follows:
[0081]
[0082] If the confidence level is within a certain threshold (a reasonable value can be obtained by testing according to actual conditions, and this embodiment does not impose any restrictions on this), it means that several consecutive frames of perception images are available, so as to avoid errors in short-distance images that lead to unavailable positioning, thereby improving the robustness of feature extraction.
[0083] After the above feature matching is completed, the subsequent operation of constructing point and line keywords to construct a bag-of-words vector is performed, as described below.
[0084] Step S30: construct keywords according to the feature point descriptors corresponding to the point features and the synchronized line features to obtain a distance dictionary and a line bag-of-words vector.
[0085] It should be noted that point features include key points and feature point descriptors around key points. Feature point descriptors provide speed vectors for bag-of-words construction.
[0086] That is, the feature point descriptor is an information representation used to describe point features, usually a vector that contains some specific attributes of the area around the feature point, such as gray value distribution characteristics, gradient characteristics, etc.
[0087] It should be noted that the distance dictionary is a dictionary constructed by the distance value information between each point feature and the line feature. When initially locating the parking vehicle, the distance dictionary can be used to quickly find the nearest line feature.
[0088] It should be noted that the line bag-of-words vector is a representation method of converting line features into vector descriptions in the process of building a bag-of-words model.
[0089] After the line features are extracted, quantified, and processed, they are represented as a vector, which is similar to the representation of text words in the bag-of-words model. It can be used to classify, identify, match, and other operations on line features to quickly retrieve the position information of the line features in the perceived image.
[0090] Specifically, the process of keyword construction may include: first, the point features and line features may be annotated with names to obtain keywords representing the point features and line features. Then, the feature point descriptors corresponding to each point feature and the synchronized line features are calculated to obtain each distance value; then, each distance value and each keyword are used to form a distance dictionary. Then, after the line features are extracted, quantized, etc., they are represented as a vector to obtain a line bag-of-words vector.
[0091] In this implementation, by constructing a distance dictionary and a line bag-of-words vector, different features are more accurately distinguished for comparison and identification in image matching.
[0092] Step S40: constructing a bag-of-words model corresponding to the vehicle according to the distance dictionary and the line bag-of-words vector.
[0093] It should be noted that the Bag of Words Model is a model in which the text in the bag of words vector and the distance dictionary are represented as vectors.
[0094] In this embodiment, in the bag-of-words model, the vector represents the feature representation of the text under the bag-of-words model, and can quickly perform text classification and information retrieval to achieve the initial positioning function of parking.
[0095] In the technical solution provided in this embodiment, the construction device can use the deep learning framework superpoint to extract feature points, and use the semantic segmentation model segnet to extract line features. Then the point features and line features are processed synchronously to obtain synchronized line features to ensure that the extracted points and lines are not noise points, so that more features are on the line surface. Then, keywords are constructed based on the feature point descriptors corresponding to the point features and the synchronized line features to obtain the distance dictionary and line bag-of-words vectors to construct the bag-of-words model corresponding to the vehicle. Since this embodiment obtains point features and line features when extracting features from the perceived image, and synchronizes the points and lines to obtain synchronized line features to construct a bag-of-words model, it can better describe horizontal and vertical structured scenes such as garages, improve the quality of bags of words in structured scenes such as garages, and thus improve the success rate of vehicle positioning initialization in garages.
[0096] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above-mentioned embodiment 1 can be referred to the above introduction, and will not be repeated in the following. Figure 1 and Figure 3 , Figure 3 A flowchart diagram of the second embodiment of the bag-of-words model construction method of the present application is provided.
[0097] In this example, step S30 includes steps S31 to S34:
[0098] Step S31: determining a feature point descriptor of the point feature, wherein the point feature includes a key point and a feature point descriptor within a preset range of the key point.
[0099] It should be noted that the key point is a representative and unique point in the perceived image, such as a corner point, etc. The preset range is a preset range of feature point descriptors used to determine the area around the key point.
[0100] Step S32: clustering the feature point descriptors and the synchronized line features to obtain a plurality of distance data.
[0101] Step S33: constructing a distance dictionary according to the plurality of distance data.
[0102] Step S34: constructing a line bag-of-words vector according to the center point and line coordinates of the synchronized line features.
[0103] Specifically, the feature point descriptors and line features can be clustered first, and the distance can be calculated to build a distance dictionary for later retrieval. Then, the feature line descriptor is used to take Li(ai, bi, ci, si, ei) to calculate the center point mi = (si + ei) / 2; at the same time, si, ei, and mi are taken to describe the line features and saved as the line word bag vector WL.
[0104] In this implementation, an accurate distance dictionary can be obtained through clustering.
[0105] In a feasible implementation, step S40 of this example includes the steps of: identifying and sorting the line bag-of-words vectors to obtain the sorted line bag-of-words vectors; recording the posture information corresponding to the line bag-of-words vectors when the vehicle is performing simultaneous positioning and map construction; and constructing a bag-of-words model corresponding to the vehicle based on the distance dictionary, the line bag-of-words vectors and the posture information.
[0106] It should be noted that simultaneous localization and mapping (SLAM) is the process in which a vehicle obtains environmental information through sensors (such as lidar, cameras, etc.) when it is in an unknown parking environment, and simultaneously locates itself and builds an environmental map.
[0107] It should be noted that the pose information is the relevant information of the position (including coordinates, etc.) and posture (such as direction, angle, etc.) of the line features in the perceived image obtained in the SLAM link.
[0108] In this embodiment, during the WL construction process, each WL will record the current posture information in the SLAM link, and then perform identification sorting to obtain the sorted line word bag vector sorting. Therefore, the corresponding posture information can be quickly retrieved in the word bag model according to the sorting.
[0109] Based on the above embodiments of the present application, in the third embodiment of the present application, the same or similar contents as those in the above embodiments can be referred to the above introduction and will not be repeated later.
[0110] like Figure 2 As shown, after constructing the bag-of-words vector, the initial parking position can be determined by vector comparison. After step S40 of this example, the following steps are further included: when receiving a parking instruction, searching the current frame captured by the vehicle camera according to the bag-of-words model to obtain the line bag-of-words vector closest to the current frame; retrieving the corresponding posture information according to the identifier of the line bag-of-words vector; and using the posture information as the initial parking position of the vehicle.
[0111] It should be noted that the parking instruction is a command to instruct the vehicle to park or park. When performing automatic parking, the current frame of the current parking environment is collected through the vehicle camera. The current frame is searched in the constructed bag-of-words model to quickly obtain the pose information of the current frame.
[0112] Specifically, the keyword closest to the current frame can be found in the distance dictionary, and then the key points similar to the current frame can be determined by the keyword; finally, the ID of the WL is located according to the key points and the corresponding posture information is quickly retrieved as the initial positioning of SLAM.
[0113] In the technical solution provided in this embodiment, the success rate of positioning initialization in a structured scenario such as a garage can be improved through retrieval using a bag-of-words model.
[0114] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the method for constructing the bag-of-words model of the present application. More simple transformations based on this technical concept are all within the scope of protection of the present application.
[0115] This application also provides a bag-of-words model construction device, please refer to Figure 4 , Figure 4 This is a schematic diagram of the module structure of a bag-of-words model construction device according to an embodiment of the present application; the bag-of-words model construction device comprises:
[0116] A feature extraction module 501 is used to extract features from the perception images collected during the parking process of the vehicle to obtain point features and line features;
[0117] A point-line synchronization module 502 is used to synchronize the point features with the line features to obtain synchronized line features;
[0118] A keyword construction module 503 is used to construct keywords according to the feature point descriptors corresponding to the point features and the synchronized line features to obtain a distance dictionary and a line bag-of-words vector;
[0119] The bag-of-words model module 504 is used to construct a bag-of-words model corresponding to the vehicle according to the distance dictionary and the line bag-of-words vector.
[0120] In this embodiment, the feature extraction module 501 is also used to obtain a perceived image collected by the vehicle during the parking process; extract feature points of the perceived image through a preset deep learning framework to obtain point features of the perceived image; and extract feature lines of the perceived image through a preset semantic segmentation model to obtain line features of the perceived image.
[0121] Furthermore, the point-line synchronization module 502 is also used to determine the point coordinates of the point feature and the line coordinates of the line feature; perform distance calculation based on the point coordinates and the line coordinates to obtain a distance value; when the distance value is less than a preset distance threshold and the point coordinates and the line coordinates meet preset coordinate conditions, project the point feature onto the corresponding line feature to obtain a synchronized line feature.
[0122] Furthermore, the keyword construction module 503 is also used to determine the feature point descriptor of the point feature, the point feature including the key point and the feature point descriptor within a preset range of the key point; cluster the feature point descriptor and the synchronized line feature to obtain multiple distance data; construct a distance dictionary based on the multiple distance data; and construct a line bag-of-words vector based on the center point and line coordinates of the synchronized line feature.
[0123] Furthermore, the bag-of-words model module 504 is also used to identify and sort the line bag-of-words vectors to obtain sorted line bag-of-words vectors; when the vehicle is performing simultaneous positioning and map construction, the posture information corresponding to the line bag-of-words vectors is recorded; and according to the distance dictionary, the line bag-of-words vectors and the posture information, the bag-of-words model corresponding to the vehicle is constructed.
[0124] Furthermore, the bag-of-words model construction device also includes an initial positioning module, which is used to retrieve the current frame captured by the vehicle camera according to the bag-of-words model when receiving a parking instruction, and obtain the line bag-of-words vector closest to the current frame; retrieve the corresponding posture information according to the identification of the line bag-of-words vector; and use the posture information as the initial positioning of the vehicle for parking.
[0125] Other embodiments or specific implementation methods of the word bag model construction device of the present application can refer to the above-mentioned method embodiments, which will not be repeated here.
[0126] The bag-of-words model construction device provided by the present application adopts the bag-of-words model construction method in the above-mentioned embodiment, which can solve the problem that the construction of traditional bag-of-words all uses feature points to train the bag-of-words, which is not very friendly to different scenes with large environmental changes and scenes with poor textures, and often has insufficient features and reduced bag-of-words performance. Technical problems. Compared with the prior art, the beneficial effects of the bag-of-words model construction device provided by the present application are the same as the beneficial effects of the bag-of-words model construction method provided by the above-mentioned embodiment, and other technical features in the bag-of-words model construction device are the same as the features disclosed in the above-mentioned embodiment method, which will not be repeated here.
[0127] The present application provides a bag-of-words model construction device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the bag-of-words model construction method in the above-mentioned embodiment 1.
[0128] Reference below Figure 5 , Figure 5 The schematic diagram of the device structure of the hardware operating environment involved in the word bag model construction method in the embodiment of the present application shows a schematic diagram of the structure of the word bag model construction device suitable for implementing the embodiment of the present application. The word bag model construction device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 5 The bag-of-words model building device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.
[0129] like Figure 5As shown, the word bag model building device may include a processing device 1001 (such as a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM: Read Only Memory) 1002 or the program loaded from the storage device 1003 to the random access memory (RAM: Random Access Memory) 1004. In the RAM 1004, various programs and data required for the operation of the word bag model building device are also stored. The processing device 1001, ROM 1002 and RAM 1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the word bag model building device to communicate with other devices wirelessly or by wire to exchange data. Although the word bag model building device with various systems is shown in the figure, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or have alternatively.
[0130] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.
[0131] The bag-of-words model construction device provided by the present application adopts the bag-of-words model construction method in the above-mentioned embodiment, which can solve the problem that the construction of traditional bag-of-words all uses feature points to train the bag-of-words, which is not very friendly to different scenes with large environmental changes and scenes with poor textures, and often has insufficient features and reduced bag-of-words performance. Technical problems. Compared with the prior art, the beneficial effects of the bag-of-words model construction device provided by the present application are the same as the beneficial effects of the bag-of-words model construction method provided by the above-mentioned embodiment, and other technical features in the bag-of-words model construction device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.
[0132] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.
[0133] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.
[0134] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, and the computer-readable program instructions are used to execute the bag-of-words model construction method in the above-mentioned embodiment.
[0135] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination of the above.
[0136] The computer-readable storage medium may be included in the bag-of-words model building device; or may exist independently without being assembled into the bag-of-words model building device.
[0137] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by the bag-of-words model building device, the bag-of-words model building device: extracts features from the perceived images collected by the vehicle during parking to obtain point features and line features; synchronizes the point features and the line features to obtain synchronized line features; constructs keywords based on the feature point descriptors corresponding to the point features and the synchronized line features to obtain a distance dictionary and a line bag-of-words vector; and constructs a bag-of-words model corresponding to the vehicle based on the distance dictionary and the line bag-of-words vector.
[0138] Computer program code for performing the operations of the present application may be written in one or more programming languages or a combination thereof, including object-oriented programming languages such as Java, Smalltalk, C++, and conventional procedural programming languages such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).
[0139] The flowchart and block diagram in the accompanying drawings illustrate the possible architecture, functions and operations of the system, method and computer program product according to various embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
[0140] The modules involved in the embodiments described in the present application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.
[0141] The readable storage medium provided by the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned word bag model construction method, which can solve the problem that the construction of traditional word bags uses feature points to train word bags, which is not very friendly to different scenes with large environmental changes and scenes with poor textures, and often has insufficient features and reduced word bag performance. Technical problems. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided by the present application are the same as the beneficial effects of the word bag model construction method provided in the above-mentioned embodiment, which will not be repeated here.
[0142] The present application also provides a computer program product, including a computer program, which implements the steps of the above-mentioned word bag model construction method when executed by a processor.
[0143] The computer program product provided by the present application can solve the technical problems that the traditional word bag construction uses feature points to train the word bag, which is not very friendly to different scenes with large environmental changes and scenes with poor textures, and often has insufficient features and reduced word bag performance. Compared with the prior art, the beneficial effects of the computer program product provided by the present application are the same as the beneficial effects of the word bag model construction method provided in the above embodiment, which will not be repeated here.
[0144] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.
Claims
1. A method for constructing a bag-of-words model, characterized in that: The method includes: Extract features from the perception images collected during the parking process to obtain point features and line features; Synchronously processing the point feature and the line feature to obtain a synchronized line feature; Keyword construction is performed according to the feature point descriptor corresponding to the point feature and the synchronized line feature to obtain a distance dictionary and a line bag-of-words vector; A bag-of-words model corresponding to the vehicle is constructed according to the distance dictionary and the line bag-of-words vector.
2. The method according to claim 1, characterized in that The step of extracting features from the perception image collected during the parking process to obtain point features and line features includes: Acquire a perception image collected by the vehicle during the parking process; Extracting feature points of the perceived image through a preset deep learning framework to obtain point features of the perceived image; Feature lines are extracted from the perceived image using a preset semantic segmentation model to obtain line features of the perceived image.
3. The method according to claim 1, characterized in that The step of synchronously processing the point features and the line features to obtain synchronized line features comprises: Determine the point coordinates of the point feature and the line coordinates of the line feature; Perform distance calculation according to the point coordinates and the line coordinates to obtain a distance value; When the distance value is less than a preset distance threshold and the point coordinates and the line coordinates meet preset coordinate conditions, the point feature is projected onto the corresponding line feature to obtain a synchronized line feature.
4. The method according to claim 1, characterized in that The step of constructing keywords according to the feature point descriptors corresponding to the point features and the synchronized line features to obtain a distance dictionary and a line bag-of-words vector comprises: Determine a feature point descriptor of the point feature, wherein the point feature includes a key point and a feature point descriptor within a preset range of the key point; Clustering the feature point descriptors and the synchronized line features to obtain a plurality of distance data; constructing a distance dictionary according to the plurality of distance data; A line bag-of-words vector is constructed according to the center point and line coordinates of the synchronized line features.
5. The method according to claim 4, characterized in that The step of constructing a bag-of-words model corresponding to the vehicle according to the distance dictionary and the line bag-of-words vector comprises: Sorting the line bag-of-words vectors by their identifiers to obtain sorted line bag-of-words vectors; When the vehicle is performing simultaneous positioning and map building, recording the position information corresponding to the line bag-of-words vector; A bag-of-words model corresponding to the vehicle is constructed according to the distance dictionary, the line bag-of-words vector and the posture information.
6. The method according to any one of claims 1 to 5, characterized in that After the step of constructing the bag-of-words model corresponding to the vehicle according to the distance dictionary and the line bag-of-words vector, the method further includes: When receiving a parking instruction, searching the current frame captured by the vehicle camera according to the bag-of-words model to obtain a line bag-of-words vector closest to the current frame; Retrieving corresponding posture information according to the identifier of the line bag-of-words vector; The position information is used as the initial positioning of the vehicle for parking.
7. A bag-of-words model construction device, characterized in that: The device comprises: A feature extraction module is used to extract features from the perception images collected during the parking process of the vehicle to obtain point features and line features; A point-line synchronization module, used for synchronously processing the point features and the line features to obtain synchronized line features; A keyword construction module, used to construct keywords according to the feature point descriptors corresponding to the point features and the synchronized line features, to obtain a distance dictionary and a line bag-of-words vector; The bag-of-words model module is used to construct a bag-of-words model corresponding to the vehicle according to the distance dictionary and the line bag-of-words vector.
8. A bag-of-words model building device, characterized in that: The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the word bag model construction method according to any one of claims 1 to 6.
9. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the word bag model construction method according to any one of claims 1 to 6 are implemented.
10. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the steps of the method for constructing a bag-of-words model according to any one of claims 1 to 6 are implemented.