Rail train sensing method based on occupied grid world model

Abstract

The invention relates to the technical field of visual perception, and provides a rail train perception method based on an occupied grid world model, and the method comprises the steps: obtaining a to-be-perceived scene image frame which comprises a current scene image frame and a historical scene image frame; extracting voxel features of the scene image frame to be sensed; splicing the category features and voxel features of the to-be-perceived scene image frames to obtain splicing features, and carrying out occupied grid scene coding on the splicing features to obtain three-dimensional scene features; and performing occupied grid world model prediction based on the three-dimensional scene features, and determining a scene perception result. According to the method provided by the invention, voxel features are extracted through a to-be-sensed scene image frame including a historical scene image frame; splicing the category features and the voxel features, and carrying out occupied grid scene coding on the spliced features to obtain compact three-dimensional scene features; the occupied grid world model is predicted based on the three-dimensional scene features, accurate and reliable scene perception is achieved, and then the safety of the rail train is improved.

Description

Technical Field

[0001] The present invention relates to the field of visual perception technology, and in particular to a rail train perception method based on an occupation grid world model. Background Art

[0002] In recent years, train perception technology has been widely explored. The maturity of algorithms such as 3D target detection and target tracking has helped trains to quickly detect abnormal targets and thus avoid obstacles. Predicting the evolution of the scene around the train in the future is crucial to the safety of train driving. Most existing train perception methods only judge the abnormal traffic participants that may exist on the train by judging the current frame, such as humans, animals, and missing tracks due to special reasons, and make braking decisions based on this.

[0003] However, judging only the current frame without making advance assumptions about possible future situations results in low accuracy and reliability of the perception results, which in turn leads to the need to improve the accuracy and rationality of the braking decisions made based on the perception results. In other words, the safety of rail trains during travel needs to be improved. Summary of the invention

[0004] The present invention provides a rail train perception method based on an occupation grid world model, so as to solve the defects of low accuracy and reliability of perception results in the prior art.

[0005] The present invention provides a rail train perception method based on an occupation grid world model, comprising: Acquire a scene image frame to be sensed, wherein the scene image frame to be sensed includes a current scene image frame and a historical scene image frame; Extracting voxel features of the image frame of the scene to be sensed; splicing the category feature of the image frame of the scene to be sensed and the voxel feature to obtain a splicing feature, and performing occupancy grid scene coding on the splicing feature to obtain a three-dimensional scene feature; An occupancy grid world model is predicted based on the three-dimensional scene features to determine a scene perception result.

[0006] According to a rail train perception method based on an occupancy grid world model provided by the present invention, the occupancy grid world model prediction is performed based on the three-dimensional scene features to determine the scene perception result, including: Splicing the three-dimensional scene features and the train posture features of the rail train to obtain world features corresponding to each image frame of the scene to be sensed; Merging each world feature multiple times to obtain multi-level world features corresponding to each world feature; Perform occupation grid world model prediction on the hierarchical world features at each level to obtain the perceptual features at each level; Aggregate the perception features at each level and predict the scene perception result.

[0007] According to a rail train perception method based on an occupancy grid world model provided by the present invention, the occupancy grid scene encoding is performed on the splicing features to obtain three-dimensional scene features, including: Performing feature conversion on the splicing features to obtain bird's-eye view features; Occupancy grid scene coding is performed on the bird's-eye view feature, and the three-dimensional scene feature is obtained based on a preset scene feature that is closest to the bird's-eye view feature.

[0008] According to a rail train perception method based on an occupancy grid world model provided by the present invention, the occupancy grid scene encoding is performed on the bird's-eye view feature, and the three-dimensional scene feature is obtained based on a preset scene feature that is closest to the bird's-eye view feature, including: Downsampling the bird's-eye view feature to obtain a low-resolution bird's-eye view feature; Performing grid scene coding on the low-resolution bird's-eye view feature, and obtaining a refined feature based on the preset scene feature that is closest to the low-resolution bird's-eye view feature; Upsampling the refined features to obtain initial resolution features; The initial resolution features are classified in the channel dimension to obtain the three-dimensional scene features.

[0009] According to a rail train perception method based on an occupancy grid world model provided by the present invention, the step of extracting voxel features of the image frame of the scene to be perceived includes: Extracting image features of the image frame of the scene to be perceived; Performing multi-scale refinement on the image features based on a feature pyramid network to obtain multi-scale image features; Extracting multi-scale voxel features based on the multi-scale image features and voxel query; The multi-scale voxel features are aggregated to obtain the voxel features.

[0010] According to a rail train perception method based on an occupancy grid world model provided by the present invention, the multi-scale voxel features are extracted based on the multi-scale image features and voxel query, including: Classifying the voxel query to obtain a non-empty voxel query; The multi-scale voxel feature is extracted by performing feature interaction with the multi-scale image feature based on the non-empty voxel query.

[0011] The present invention also provides a rail train perception device based on an occupation grid world model, comprising: An acquisition unit, which acquires a scene image frame to be sensed, wherein the scene image frame to be sensed includes a current scene image frame and a historical scene image frame; An extraction unit, for extracting voxel features of the image frame of the scene to be sensed; A sampling unit splices the category feature of the image frame of the scene to be sensed with the voxel feature to obtain a spliced ​​feature, and performs occupancy grid scene coding on the spliced ​​feature to obtain a three-dimensional scene feature; The perception unit predicts an occupancy grid world model based on the three-dimensional scene features to determine a scene perception result.

[0012] The rail train perception method based on the occupancy grid world model provided by the present invention obtains a scene image frame to be perceived through a current scene image frame and a historical scene image frame, and extracts voxel features of the scene image frame to be perceived; splices the category feature with the voxel feature to obtain a spliced ​​feature, performs occupancy grid scene encoding on the spliced ​​feature to obtain a three-dimensional scene feature; performs occupancy grid world model prediction based on the three-dimensional scene feature to determine the scene perception result, thereby achieving accurate and reliable scene perception, and thus improving the safety of rail trains. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0014] Figure 1 It is one of the flow diagrams of the rail train perception method based on the occupancy grid world model provided by the present invention; Figure 2 This is the second flow chart of the rail train perception method based on the occupancy grid world model provided by the present invention; Figure 3 It is a structural schematic diagram of a rail train sensing device based on an occupation grid world model provided by the present invention. DETAILED DESCRIPTION

[0015] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0016] With the continuous development of the field of artificial intelligence and the continuous upgrading of graphics card performance, computer vision technology has gradually matured. People realize the perception of the surrounding environment by installing visual sensors on various traffic participants such as cars, ships, and trains, so as to achieve safe and reliable driving. At present, in the field of rail trains, most of the existing train perception methods only judge the current frame. However, the accuracy and reliability of the perception results obtained based on this perception method need to be further improved. In response to this problem, the present invention provides a rail train perception method based on an occupied grid world model to achieve accurate and reliable rail train perception. Figure 1 is one of the flow charts of the rail train perception method based on the occupancy grid world model provided by the present invention, such as Figure 1 As shown, the method includes: Step 110: Acquire a scene image frame to be sensed, wherein the scene image frame to be sensed includes a current scene image frame and a historical scene image frame.

[0017] Specifically, the camera installed on the rail train can capture the image of the scene in front of the train in real time as the current scene image frame. In addition, the image captured at a historical moment can be used as the historical scene image frame. Thus, the current scene image frame acquired in real time and a certain number of historical scene image frames set in advance can be used as the scene image frames to be perceived, and the obtained scene image frames to be perceived together constitute the time series data of the train operation environment.

[0018] It should be noted that, compared with the prior art which only perceives the current scene image frame, in the embodiment of the present invention, by perceiving both the historical scene image frame and the current scene image frame, the accuracy and reliability of the perception result are greatly improved.

[0019] Step 120: extract voxel features of the image frame of the scene to be perceived.

[0020] Here, voxel features can reflect the position, color, brightness and other information of each object in the scene in front of the rail train, which helps to understand the structure and properties of the three-dimensional scene.

[0021] Specifically, the current scene image frame collected in real time can be input into the backbone network, and the image features of the current scene image frame can be extracted in real time through the backbone network. Thus, the image features of the historical scene image frame extracted at the historical moment and the image features of the current scene image frame can be used as the image features of the scene image frame to be perceived. Furthermore, by performing a 3D voxel query on each image feature, image feature aggregation can be completed in the 3D space to obtain the voxel features corresponding to each scene image frame to be perceived.

[0022] Step 130 , splicing the category feature of the image frame of the scene to be sensed and the voxel feature to obtain a spliced ​​feature, performing occupancy grid scene coding on the spliced ​​feature to obtain a three-dimensional scene feature.

[0023] Here, the category features can be used to reflect the category information of objects in the image frame of the scene to be perceived, such as tracks, pedestrians, fallen rocks, trees, etc., that is, to reflect the high-level semantic information in the scene. Therefore, the semantic information (category features) is combined with the spatial information (voxel features) to obtain the splicing features, and the splicing features reflect a more comprehensive scene description. In addition, the three-dimensional scene features here can be used to reflect the key spatial information and semantic information in each grid structure by simplifying the three-dimensional scene into an easy-to-process grid structure.

[0024] Specifically, the image frame of the scene to be perceived can be subjected to preliminary target recognition, and the category features of the image frame of the scene to be perceived can be extracted. , that is, for each object category Build a category embedding, The category features of all object categories are represented. The category features of the image frame of the scene to be perceived are concatenated with the voxel features in the height dimension to obtain a concatenated feature with more comprehensive feature information, thereby improving the computational efficiency of the perception method of the rail train. Furthermore, the concatenated features can be encoded into an occupancy grid scene by an encoder to obtain a compact three-dimensional scene feature.

[0025] It should be noted that by extracting features from the scene image frames to be perceived that include historical scene image frames, and jointly encoding the high-dimensional category features with the extracted features reflecting spatial information, more refined, compact, and higher-dimensional three-dimensional scene features are obtained, making the scene perception results obtained based on the three-dimensional scene features more accurate and reliable.

[0026] Step 140: perform occupancy grid world model prediction based on the three-dimensional scene features to determine a scene perception result.

[0027] Specifically, the three-dimensional scene features can be input into the decoder, and the occupancy grid world model prediction can be performed through the decoder to predict the world perception results corresponding to each grid, and then the scene perception results of the rail train can be obtained. The scene perception results here can be humans, animals, blank grids, etc. Further, the scene perception results of the rail train can be used to determine whether the drivable track area of ​​the rail train will have abnormal grid occupation in the future. For example, when the scene perception results can be objects such as humans and animals, it means that the drivable track area of ​​the rail train will have abnormal grid occupation in the future, and the train can be instructed to take braking measures in advance in the current frame to ensure the safe travel of the train; when the scene perception result can be a blank grid, it means that the drivable track area of ​​the rail train will not have abnormal grid occupation in the future, and the train can be instructed not to take braking measures in advance in the current frame. Therefore, by predicting the future 3D occupancy grid scene, early identification of possible dangers can be achieved, assisting train drivers to discover and solve dangers earlier, and improving the safety of rail trains during travel.

[0028] The method provided by the embodiment of the present invention obtains a scene image frame to be perceived through a current scene image frame and a historical scene image frame, and extracts voxel features of the scene image frame to be perceived; splices the category features with the voxel features to obtain spliced ​​features, and performs occupancy grid scene encoding on the spliced ​​features to obtain three-dimensional scene features; performs occupancy grid world model prediction based on the three-dimensional scene features to determine the scene perception result, thereby achieving accurate and reliable scene perception, and thereby improving the safety of rail trains.

[0029] Based on any of the above embodiments, step 140 includes: Splicing the three-dimensional scene features and the train posture features of the rail train to obtain world features corresponding to each image frame of the scene to be sensed; Merging each world feature multiple times to obtain multi-level world features corresponding to each world feature; Perform occupation grid world model prediction on the hierarchical world features at each level to obtain the perceptual features at each level; Aggregate the perception features at each level and predict the scene perception result.

[0030] Here, the train posture feature can be used to reflect the actual position, travel direction and other information of the rail train in three-dimensional space. Therefore, the world feature here can reflect the static information and dynamic information in the scene where the rail train is located. The static information is reflected through the three-dimensional scene features, and the dynamic information can be reflected through the train posture feature.

[0031] Specifically, first, the position information of the rail train can be obtained through the positioning information of the rail train. By encoding the position information of the rail train, the train position feature of the rail train is obtained, which is recorded as ,in, Indicates Then, the three-dimensional scene features and the train posture features of the track train can be spliced ​​to obtain the world features corresponding to each scene image frame to be perceived, which is recorded as ,in, Represent the length, width and height of the predefined voxel space respectively.

[0032] Then, the world features of the historical frame and the current frame are Input spatiotemporal feature sampling module, for each frame of world features The spatial feature sampling module is based on the self-attention mechanism, which realizes the interactive modeling of scene features and posture features within the world features and obtains multi-level world features. In detail, each world feature can be merged multiple times. For example, each world feature can be merged with a step of 2. The 3D scene features within the range are reduced by 4 times, and the module repeats this step. This gives us multi-level world features. ,in, Indicates The world features correspond to Layered world features.

[0033] Furthermore, the multi-level world features corresponding to all the world features can be divided into levels to obtain the hierarchical world features at each level. Then, the occupancy grid world model is predicted for the hierarchical world features at each level to obtain the perceptual features at each level, which are recorded as ,in, Indicates Hierarchical perception features. For example, multiple sub-prediction models can be used Predict scene changes of different scales and obtain perceptual features at all levels. Represents the sub-prediction model used to predict the first level.

[0034] It should be noted that for each sub-prediction model , at each location Complete the time attention, fuse the time series information, and obtain the hierarchical perception features of the fused time series information at each scale, thereby improving the accuracy and reliability of the scene perception prediction results. It should also be noted that by merging the features of each world feature multiple times, the multi-level world features corresponding to each world feature are obtained. Then, the hierarchical world features at each level are predicted using the occupancy grid world model to obtain the perception features at each level, that is, the autoregressive pyramid strategy is used to gradually and progressively predict the future scene representation, thereby improving the accuracy and reliability of the scene perception prediction results.

[0035] Finally, the future world features can be obtained by aggregating the perceptual features at each level Then, the future world features can be input into the pose decoder and the scene decoder to obtain the train pose output and the scene prediction output, that is, the scene perception result. Here, the pose output can be calculated by the following formula, as shown below:

[0036] In the formula, Indicates Train position output at the time; Represents the pose decoder.

[0037] In addition, the scene prediction output here can be calculated by the following formula, as shown below:

[0038] In the formula, Indicates The scene prediction output at the moment; Represents a scene decoder.

[0039] The method provided by the embodiment of the present invention performs multiple feature merging on each world feature to obtain multi-level world features corresponding to each world feature; performs occupancy grid world model prediction on the hierarchical world features at each level to obtain perception features at each level, that is, by adopting the autoregressive pyramid strategy to gradually and progressively predict future scene representations, using the train posture decoder to obtain the train driving status, and using the scene decoder to obtain the track driving area status, accurate and reliable scene perception results are obtained. In addition, the feature prediction method from coarse to fine greatly reduces the space occupation while improving the operating efficiency of the track train perception method.

[0040] Based on any of the above embodiments, in step 130, performing grid scene coding on the splicing features to obtain three-dimensional scene features includes: Performing feature conversion on the splicing features to obtain bird's-eye view features; Occupancy grid scene coding is performed on the bird's-eye view feature, and the three-dimensional scene feature is obtained based on a preset scene feature that is closest to the bird's-eye view feature.

[0041] Specifically, the voxel feature and the category feature can be spliced ​​in the height dimension to obtain the spliced ​​feature. Convert features into two-dimensional bird's-eye view features Then, the occupancy grid scene encoding can be performed on the bird's-eye view features. The encoder here can be ,in Defines the number of codes, D represents the height of the voxel space, C represents the object category, and each code A high-level feature expression corresponding to the 3D scene, such as whether there is an animal at the corresponding location. Then, the bird's-eye view feature Each spatial feature Convert to the preset scene feature with the closest distance. Here, the preset scene feature with the closest distance corresponding to each spatial feature can be obtained by the following formula, as shown in the following formula:

[0042] In the formula, Representing spatial features Corresponding preset scene features; Represents the occupancy grid scene encoder; Indicates the preset scene characteristics; express distance.

[0043] Finally, the preset scene features with the closest distance corresponding to each spatial feature are aggregated to obtain the three-dimensional scene feature , that is, we get a feature representation that can indicate what kind of object exists at each occupied grid.

[0044] Based on any of the above embodiments, performing occupation grid scene coding on the bird's-eye view feature, and obtaining the three-dimensional scene feature based on a preset scene feature that is closest to the bird's-eye view feature, includes: Downsampling the bird's-eye view feature to obtain a low-resolution bird's-eye view feature; Performing grid scene coding on the low-resolution bird's-eye view feature, and obtaining a refined feature based on the preset scene feature that is closest to the low-resolution bird's-eye view feature; Upsampling the refined features to obtain initial resolution features; The initial resolution features are classified in the channel dimension to obtain the three-dimensional scene features.

[0045] Specifically, the bird's-eye view features can be downsampled through a 2D convolutional layer: , and obtain low-resolution bird's-eye view features. ,in represents a predefined downsampling factor, Represents a predefined orbital scene object category.

[0046] Furthermore, in order to further refine the features and obtain a denser 3D scene representation, an occupancy grid scene encoder can be designed , the low-resolution bird's-eye view feature is occupies a grid scene encoding, and a refined feature is obtained based on the preset scene feature closest to the low-resolution bird's-eye view feature. The refined feature is upsampled and converted to the original resolution to obtain the initial resolution feature. Here, upsampling can be achieved by the following formula, as shown in the following formula:

[0047] In the formula, represents the initial resolution feature; Represents refinement features.

[0048] Finally, the initial resolution features are segmented in the channel dimension, and then the segmented refined features are classified based on the softmax function to obtain the three-dimensional scene features and realize the 3D feature height dimensional reconstruction. Here, the three-dimensional scene features can be obtained by the following formula, as shown in the following formula:

[0049] In the formula, Represents the refined features after segmentation; Represents the characteristics of a three-dimensional scene.

[0050] Based on any of the above embodiments, step 120 includes: Extracting image features of the image frame of the scene to be perceived; Performing multi-scale refinement on the image features based on a feature pyramid network to obtain multi-scale image features; Extracting multi-scale voxel features based on the multi-scale image features and voxel query; The multi-scale voxel features are aggregated to obtain the voxel features.

[0051] Specifically, the image features of each scene image frame to be perceived can be extracted through an image backbone network, such as ResNet and SwinTransformer network, which are recorded as ,in Respectively represent the length and width of the scene image frame to be perceived. Then, the image features can be multi-scale refined through the feature pyramid network to obtain multi-scale image features, which are recorded as ,in, .

[0052] Next, the multi-scale image features and voxel queries can be input into the voxel cross attention network to extract the multi-scale voxel features, denoted as ,in , in order to achieve the extraction of 3D voxel occupancy features from coarse to fine, so that the extracted voxel features are more accurate. For example, it can be achieved by the following formula, as shown in the following formula:

[0053] In the formula, represents a voxel query; represents multi-scale image features, Indicates scale; Represents the number of attention heads in the voxel cross-attention module; It is the attention head The weight of the value corresponding to the query, For each query The number of interactive sampling with image features; Represents the optimized query The normalized attention weights are ; Indicates that the subquery To value The transfer matrix of Indicates The attention head The sampling points are relative to the reference point The position offset, Indicates that it is located in the image feature Image features of the location.

[0054] Furthermore, multi-scale voxel features can be fused to obtain fine-grained voxel features. In detail, multi-scale voxel features are aggregated from top to bottom: , and so on until you get .

[0055] The method provided by the embodiment of the present invention obtains multi-scale image features by performing multi-scale refinement on the image features based on a feature pyramid network; extracts multi-scale voxel features based on multi-scale image features and voxel queries; and aggregates the multi-scale voxel features to obtain voxel features, thereby realizing coarse-to-fine 3D voxel feature extraction, improving the accuracy of the extracted voxel features, greatly reducing space occupancy, and improving model operation efficiency.

[0056] It should be noted that if the voxel query is directly interacted with the multi-scale image features, a large amount of invalid calculation and storage space consumption will be caused due to the inevitable empty grids in the voxel space. In order to further improve the computing efficiency and the utilization of storage space, based on any of the above embodiments, the multi-scale voxel features are extracted based on the multi-scale image features and voxel query, including: Classifying the voxel query to obtain a non-empty voxel query; The multi-scale voxel feature is extracted by performing feature interaction with the multi-scale image feature based on the non-empty voxel query.

[0057] Specifically, the voxel query can be Input into a binary classifier to determine whether the voxel position is an empty voxel. The binary classifier here is trained with true values. Then, to ensure efficient calculation, the former in the binary classifier can be selected. Non-empty voxel query To complete the subsequent operations of interacting with multi-scale image features, the Transformer architecture is used to extract multi-scale voxel features.

[0058] Based on any of the above embodiments, Figure 2 This is the second flow chart of the rail train perception method based on the occupancy grid world model provided by the present invention, such as Figure 2 As shown, the method includes: Step 210, occupancy grid representation acquisition: using the backbone network to extract features of the real-time collected images of the train, further designing 3D voxel query based on the cross attention mechanism, and completing image feature aggregation in 3D space; Step 220, 3D occupancy grid scene sampling: To facilitate subsequent modeling, the 3D occupancy grid representation is jointly encoded with the category embedding to obtain a compact scene representation, and then up-sampled based on the 2D deconvolution layer to complete high-level feature extraction; Step 230, spatiotemporal feature sampling and decoding: the temporal 3D occupancy grid representation is fused and interacted with the train posture information, and the autoregressive pyramid strategy is used to gradually and progressively predict the future scene representation. Furthermore, the train posture decoder is used to obtain the train driving status, and the scene decoder is used to obtain the track driving area status; Among them, during the training phase, the 3D occupancy scene sampling module and the decoder can be trained by calculating the world model loss Perform model parameter iteration. The world model loss here can be expressed by the following formula, as shown below:

[0059] In the formula, represents the world model loss; represents the scene perception result predicted by the occupancy grid world model; Indicates the scene label.

[0060] Step 240, train driving decision: complete the driving decision according to the 3D occupancy scene and the train posture prediction results to ensure the safety of train driving.

[0061] Based on any of the above embodiments, Figure 3 is a schematic diagram of the structure of a rail train sensing device based on an occupation grid world model provided by the present invention, such as Figure 3 As shown, the device comprises: An acquisition unit 310 acquires a scene image frame to be sensed, wherein the scene image frame to be sensed includes a current scene image frame and a historical scene image frame; An extraction unit 320 extracts voxel features of the image frame of the scene to be sensed; The sampling unit 330 splices the category feature of the image frame of the scene to be sensed with the voxel feature to obtain a spliced ​​feature, and performs occupancy grid scene coding on the spliced ​​feature to obtain a three-dimensional scene feature; The perception unit 340 performs occupancy grid world model prediction based on the three-dimensional scene features to determine a scene perception result.

[0062] The device provided by the embodiment of the present invention obtains a scene image frame to be perceived through a current scene image frame and a historical scene image frame, and extracts voxel features of the scene image frame to be perceived; splices the category feature with the voxel feature to obtain a spliced ​​feature, occupies a grid scene encoding the spliced ​​feature to obtain a three-dimensional scene feature; performs an occupancy grid world model prediction based on the three-dimensional scene feature to determine the scene perception result, thereby achieving accurate and reliable scene perception, and thereby improving the safety of rail trains.

[0063] Based on any of the foregoing embodiments, the sensing unit is specifically used for: Splicing the three-dimensional scene features and the train posture features of the rail train to obtain world features corresponding to each image frame of the scene to be sensed; Merging each world feature multiple times to obtain multi-level world features corresponding to each world feature; Perform occupation grid world model prediction on the hierarchical world features at each level to obtain the perceptual features at each level; Aggregate the perception features at each level and predict the scene perception result.

[0064] Based on any of the above embodiments, the sampling unit is specifically used for: Performing feature conversion on the splicing features to obtain bird's-eye view features; Occupancy grid scene coding is performed on the bird's-eye view feature, and the three-dimensional scene feature is obtained based on a preset scene feature that is closest to the bird's-eye view feature.

[0065] Based on any of the above embodiments, the sampling unit is further specifically used for: Downsampling the bird's-eye view feature to obtain a low-resolution bird's-eye view feature; Performing grid scene coding on the low-resolution bird's-eye view feature, and obtaining a refined feature based on the preset scene feature that is closest to the low-resolution bird's-eye view feature; Upsampling the refined features to obtain initial resolution features; The initial resolution features are classified in the channel dimension to obtain the three-dimensional scene features.

[0066] Based on any of the above embodiments, the extraction unit is specifically used for: Extracting image features of the image frame of the scene to be perceived; Performing multi-scale refinement on the image features based on a feature pyramid network to obtain multi-scale image features; Extracting multi-scale voxel features based on the multi-scale image features and voxel query; The multi-scale voxel features are aggregated to obtain the voxel features.

[0067] Based on any of the above embodiments, the extraction unit is further specifically used for: Classifying the voxel query to obtain a non-empty voxel query; The multi-scale voxel feature is extracted by performing feature interaction with the multi-scale image feature based on the non-empty voxel query.

[0068] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0069] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0070] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A rail train perception method based on an occupation grid world model, characterized in that: include: Acquire a scene image frame to be sensed, wherein the scene image frame to be sensed includes a current scene image frame and a historical scene image frame; Extracting voxel features of the image frame of the scene to be sensed; splicing the category feature of the image frame of the scene to be sensed and the voxel feature to obtain a splicing feature, and performing occupancy grid scene coding on the splicing feature to obtain a three-dimensional scene feature; An occupancy grid world model is predicted based on the three-dimensional scene features to determine a scene perception result.

2. The rail train perception method based on the occupancy grid world model according to claim 1 is characterized in that: The performing occupancy grid world model prediction based on the three-dimensional scene features to determine the scene perception result includes: Splicing the three-dimensional scene features and the train posture features of the rail train to obtain world features corresponding to each image frame of the scene to be sensed; Merging each world feature multiple times to obtain multi-level world features corresponding to each world feature; Perform occupation grid world model prediction on the hierarchical world features at each level to obtain the perceptual features at each level; Aggregate the perception features at each level and predict the scene perception result.

3. The rail train perception method based on the occupancy grid world model according to claim 1, characterized in that: The step of performing grid scene coding on the splicing features to obtain three-dimensional scene features includes: Performing feature conversion on the splicing features to obtain bird's-eye view features; Occupancy grid scene coding is performed on the bird's-eye view feature, and the three-dimensional scene feature is obtained based on a preset scene feature that is closest to the bird's-eye view feature.

4. The rail train perception method based on the occupancy grid world model according to claim 3 is characterized in that: The performing grid scene coding on the bird's-eye view feature to obtain the three-dimensional scene feature based on a preset scene feature that is closest to the bird's-eye view feature includes: Downsampling the bird's-eye view feature to obtain a low-resolution bird's-eye view feature; Performing grid scene coding on the low-resolution bird's-eye view feature, and obtaining a refined feature based on the preset scene feature that is closest to the low-resolution bird's-eye view feature; Upsampling the refined features to obtain initial resolution features; The initial resolution features are classified in the channel dimension to obtain the three-dimensional scene features.

5. The rail train perception method based on the occupancy grid world model according to any one of claims 1 to 3, characterized in that: The step of extracting voxel features of the scene image frame to be sensed includes: Extracting image features of the image frame of the scene to be perceived; Performing multi-scale refinement on the image features based on a feature pyramid network to obtain multi-scale image features; Extracting multi-scale voxel features based on the multi-scale image features and voxel query; The multi-scale voxel features are aggregated to obtain the voxel features.

6. The rail train perception method based on the occupancy grid world model according to claim 5, characterized in that: The extracting of multi-scale voxel features based on the multi-scale image features and voxel query includes: Classifying the voxel query to obtain a non-empty voxel query; The multi-scale voxel feature is extracted by performing feature interaction with the multi-scale image feature based on the non-empty voxel query.

7. A rail train perception device based on an occupation grid world model, characterized in that: include: An acquisition unit, which acquires a scene image frame to be sensed, wherein the scene image frame to be sensed includes a current scene image frame and a historical scene image frame; An extraction unit, for extracting voxel features of the image frame of the scene to be sensed; A sampling unit splices the category feature of the image frame of the scene to be sensed with the voxel feature to obtain a spliced ​​feature, and performs occupancy grid scene coding on the spliced ​​feature to obtain a three-dimensional scene feature; The perception unit predicts an occupancy grid world model based on the three-dimensional scene features to determine a scene perception result.

Images