Road scene determination method and device, storage medium and electronic device

Abstract

The application discloses a kind of determination method and device of road scene, storage medium and electronic device, wherein the above-mentioned method includes: using image acquisition device to the road environment where vehicle is located carries out image acquisition, obtains road scene image;Road scene image is input into scene identification model, obtains the scene identification result that scene identification model outputs, wherein scene identification model includes: encoder module, first attention module, decoder module, second attention module, output module, encoder module is used to obtain the encoding image that road scene image is encoded, first attention module is used to obtain the feature image that encoding image is extracted, decoder module is used to obtain the decoding image that feature image is decoded, second attention module is used to the fusion processing that encoding image and decoding image are handled, obtain feature fusion image, output module is used to identify feature fusion image and obtain scene identification result.

Description

Technical Field

[0001] This application relates to the field of autonomous driving, and more specifically, to a method and apparatus for determining road scenes, a storage medium, and an electronic device. Background Technology

[0002] Currently, autonomous driving technology is constantly developing. In autonomous driving, the perception of the driving environment is very important. Existing scene recognition models for perceiving the environment usually use group convolution to allow the weights of different layers to learn more diverse features, and combine spatial pyramids and feature pyramids to fuse features at different semantic levels. However, the above models have complex structures, large computational loads, and low recognition efficiency.

[0003] Regarding the relevant technologies, there is currently no effective solution to the problem of how to improve the efficiency of vehicle recognition of road scenes.

[0004] Therefore, it is necessary to improve the relevant technology to overcome the aforementioned defects. Summary of the Invention

[0005] This application provides a method and apparatus for determining road scenes, a storage medium, and an electronic device to at least address the problem of how to improve the efficiency of vehicle recognition of road scenes.

[0006] According to one aspect of the embodiments of this application, a method for determining a road scene is provided, comprising: using an image acquisition device to acquire images of the road environment in which a vehicle is located, thereby obtaining a road scene image; inputting the road scene image into a scene recognition model, thereby obtaining a scene recognition result output by the scene recognition model, wherein the scene recognition model comprises: an encoder module, a first attention module, a decoder module, a second attention module, and an output module, wherein the encoder module is used to acquire an encoded image obtained by encoding the road scene image, the first attention module is used to acquire a feature image obtained by feature extraction from the encoded image, the decoder module is used to acquire a decoded image obtained by decoding the feature image, the second attention module is used to perform fusion processing on the encoded image and the decoded image to obtain a feature fusion image, and the output module is used to recognize the feature fusion image to obtain the scene recognition result.

[0007] In an exemplary embodiment, the encoded image obtained by the encoder encoding the road scene image is acquired by: normalizing the road scene image according to a preset format, and inputting the normalized road scene image into the encoder module; downsampling the normalized road scene image by the encoder module to obtain the encoded image, wherein the resolution of the encoded image is smaller than the resolution of the road scene image, and the encoded image has image features with different sampling sizes.

[0008] In an exemplary embodiment, the feature image obtained by the first attention module performing feature extraction on the encoded image is obtained by: inputting the encoded image into the first attention module; performing dimensionality reduction processing on the feature dimension of the encoded image using a first convolutional kernel in the first attention module, wherein the feature dimension of the dimensionality-reduced encoded image is smaller than the feature dimension of the encoded image before dimensionality reduction; performing global feature learning on the dimensionality-reduced encoded image based on the nonlinear activation function of the first attention module to obtain an initial feature image; and performing dimensionality increase processing on the feature dimension of the initial feature image using a second convolutional kernel in the first attention module to obtain the feature image, wherein the feature dimension of the feature image is larger than the feature dimension of the initial feature image.

[0009] In an exemplary embodiment, the decoded image obtained by the decoder module decoding the feature image is obtained by: inputting the feature image into the decoder module, and upsampling the feature image by the decoder module to obtain the decoded image, wherein the resolution of the decoded image is greater than the resolution of the feature image.

[0010] In an exemplary embodiment, the feature fusion image obtained by the second attention module fusing the encoded image and the decoded image is obtained by the following method: inputting the encoded image and the decoded image into the second attention module, extracting features from the decoded image using a third convolutional kernel in the second attention module to obtain local features of the decoded image, and extracting features from the decoded image using a fourth convolutional kernel in the second attention module to obtain global features of the decoded image; and performing global average pooling on the local features, the global features, and the encoded image to obtain the feature fusion image.

[0011] In an exemplary embodiment, the scene recognition result obtained by the plurality of output modules recognizing the feature fusion image is acquired by: inputting the feature fusion image into the output module to obtain a plurality of recognition results, wherein the output module includes a plurality of output sub-modules, each output sub-module corresponding to a scene recognition task, the scene recognition task including at least: lane line recognition task, drivable area recognition task, and target detection recognition task; and determining the plurality of recognition results as the scene recognition result, wherein the plurality of recognition results including at least lane line recognition result, drivable area recognition result, and target detection recognition result.

[0012] In an exemplary embodiment, the scene recognition model is trained as follows: a training image set and a test image set are obtained from different types of road scene images provided in an image library, wherein the road scene images in the image library are all labeled with image recognition results; the initial model is trained using the training image set as input samples and the image recognition results as output samples; the initial model is tested using the test image set, and training is stopped when the image recognition accuracy of the initial model is greater than a preset threshold, thereby obtaining the scene recognition model.

[0013] According to another aspect of the embodiments of this application, a road scene determination device is also provided, comprising: an image acquisition module, used to acquire an image of the road scene where the vehicle is located using an image acquisition device to obtain a road scene image; and a scene recognition module, used to input the road scene image into a scene recognition model to obtain a scene recognition result output by the scene recognition model, wherein the scene recognition model comprises: an encoder module, a first attention module, a decoder module, a second attention module, and an output module, wherein the encoder module is used to acquire an encoded image obtained by encoding the road scene image, the first attention module is used to acquire a feature image obtained by feature extraction from the encoded image, the decoder module is used to acquire a decoded image obtained by decoding the feature image, the second attention module is used to perform fusion processing on the encoded image and the decoded image to obtain a feature fusion image, and the output module is used to recognize the feature fusion image to obtain the scene recognition result.

[0014] According to another aspect of the embodiments of this application, a computer-readable storage medium is also provided, wherein a computer program is stored in the computer-readable storage medium, and the computer program is configured to execute the above-described method for determining the road scenario when it is run.

[0015] According to another aspect of the embodiments of this application, an electronic device is also provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the method for determining the road scene through the computer program.

[0016] This application enables the acquisition of road scene images from vehicle road environments. Scene recognition models then perform encoding, feature extraction, decoding, and image fusion on these images to obtain scene recognition results. This approach reduces the computational load of the image recognition process while improving scene image recognition accuracy, thereby accelerating scene image recognition efficiency and enhancing the safety and response speed of autonomous driving. Attached Figure Description

[0017] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments of this application and, together with the description thereof, serve to explain this application and do not constitute an undue limitation thereof. In the drawings:

[0018] Figure 1 This is a hardware structure block diagram of a computer terminal that executes the method for determining a road scene according to an embodiment of this application;

[0019] Figure 2 This is a flowchart of a method for determining a road scene according to an embodiment of this application;

[0020] Figure 3 This is a schematic diagram of the structure of a scene recognition model according to an embodiment of this application;

[0021] Figure 4 This is a schematic diagram of the structure of the first attention module of the scene recognition model according to an embodiment of this application;

[0022] Figure 5 This is a schematic diagram of the structure of the second attention module of the scene recognition model according to an embodiment of this application;

[0023] Figure 6 This is a structural block diagram of a road scene determination device according to an embodiment of this application. Detailed Implementation

[0024] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0025] It should be noted that the terms and terms such as "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0026] The methods and embodiments provided in this application can be executed on a computer terminal or similar computing device. Taking running on a computer terminal as an example, Figure 1 This is a hardware structure block diagram of a computer terminal executing the road scene determination method according to embodiments of this application. For example... Figure 1 As shown, a computer terminal may include one or more ( Figure 1 Only one is shown in the diagram. The processor 102 (which may include, but is not limited to, a microprocessor unit (MPU) or a programmable logic device (PLD)) and a memory 104 configured to store data are also included. In one exemplary embodiment, the computer terminal may further include a transmission device 106 configured for communication and an input / output device 108. Those skilled in the art will understand that… Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the computer terminal described above. For example, the computer terminal may also include components that are more complex than those described above. Figure 1 The more or fewer components shown, or having the same Figure 1 Equivalent functions or ratios shown Figure 1 The functions shown have more different configurations.

[0027] The memory 104 may be configured to store computer programs, such as application software programs and modules, like the computer program corresponding to the road scene determination method in this embodiment. The processor 102 executes various functional applications and data processing by running the computer programs stored in the memory 104, thereby implementing the aforementioned method. The memory 104 may include high-speed random access memory and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to a computer terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0028] The transmission device 106 is configured to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by a communication provider for the computer terminal. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module configured to communicate with the Internet wirelessly.

[0029] This embodiment provides a method for determining a road scene. Figure 2 This is a flowchart of a method for determining a road scene according to an embodiment of this application, which includes the following steps:

[0030] Step S202: Use an image acquisition device to acquire images of the road environment where the vehicle is located, and obtain a road scene image;

[0031] Optionally, in step S202 above, for example, the image acquisition device includes a forward-facing camera of the vehicle, and the road scene image represents the road on which the vehicle is traveling and the surrounding environment. The image includes traffic signs, vehicles, pedestrians, environmental features, etc.

[0032] Step S204: Input the road scene image into the scene recognition model to obtain the scene recognition result output by the scene recognition model. The scene recognition model includes: an encoder module, a first attention module, a decoder module, a second attention module, and an output module. The encoder module is used to acquire an encoded image obtained by encoding the road scene image. The first attention module is used to acquire a feature image obtained by extracting features from the encoded image. The decoder module is used to acquire a decoded image obtained by decoding the feature image. The second attention module is used to fuse the encoded image and the decoded image to obtain a feature fusion image. The output module is used to recognize the feature fusion image to obtain the scene recognition result.

[0033] Optionally, in step S204 above, the scene recognition model may include more or fewer modules, or include modules with equivalent functions to the modules mentioned above.

[0034] Through the above steps, road scene images can be acquired by capturing images of the vehicle's road environment. These images are then processed using a scene recognition model, including encoding, feature extraction, decoding, and image fusion, to obtain scene recognition results. This approach reduces the computational load of the image recognition process while improving scene image recognition accuracy, thereby accelerating scene image recognition efficiency and enhancing the safety and response speed of autonomous driving. It solves the problem of how to improve the vehicle's efficiency in recognizing road scenes, thus achieving the desired effect of improved vehicle-to-road scene recognition efficiency.

[0035] In an exemplary embodiment, in step S204 above, the encoded image obtained by the encoder encoding the road scene image can be obtained by: normalizing the road scene image according to a preset format, and inputting the normalized road scene image into the encoder module; downsampling the normalized road scene image through the encoder module to obtain the encoded image, wherein the resolution of the encoded image is smaller than the resolution of the road scene image, and the encoded image has image features with different sampling sizes.

[0036] Optionally, in the above embodiments, the ESPNet network (Efficient Spatial Pyramid Convolutional Neural Network for Semantic Segmentation) can be used as the encoder. ESPNet is a lightweight convolutional neural network that improves image processing performance and efficiency by introducing an effective spatial pyramid structure, dilated convolution, and attention mechanism.

[0037] Optionally, in the above embodiments, the road scene image is normalized according to a preset format. For example, the road scene image is processed into a normalized RGB image of (H, W, C), where H represents the image height, W represents the image width, and C represents the number of image channels. Typically, the number of image channels for a color image is 3, and the number of image channels for a grayscale image is 1.

[0038] Optionally, in the above embodiments, the process of downsampling the road scene image can be repeated multiple times. For example, the road scene image with input format (H, W, 3) can be downsampled 5 times to reduce the resolution of the image by a factor of 32, resulting in an coded image with output format (C, H / 32, W / 32).

[0039] In an exemplary embodiment, in step S204 above, the feature image obtained by the first attention module performing feature extraction on the encoded image can be obtained in the following manner: the encoded image is input to the first attention module, and the feature dimension of the encoded image is reduced by the first convolution kernel in the first attention module, wherein the feature dimension of the encoded image after dimensionality reduction is smaller than the feature dimension of the encoded image before dimensionality reduction; global feature learning is performed on the dimensionality-reduced encoded image based on the nonlinear activation function of the first attention module to obtain an initial feature image; the feature dimension of the initial feature image is increased by the second convolution kernel in the first attention module to obtain the feature image, wherein the feature dimension of the feature image is larger than the feature dimension of the initial feature image.

[0040] Optionally, in the above embodiments, image dimensionality reduction can remove redundant information and retain important features, making the image features more compact. Therefore, performing global feature learning on the image after dimensionality reduction and then performing dimensionality increase on the image after completing global feature learning can reduce the amount of computation, speed up the feature learning process, and thus improve the recognition efficiency of the scene model.

[0041] In an exemplary embodiment, in step S204 above, the decoded image obtained by the decoder module decoding the feature image can be obtained by: inputting the feature image into the decoder module, upsampling the feature image by the decoder module to obtain the decoded image, wherein the resolution of the decoded image is greater than the resolution of the feature image.

[0042] Optionally, in the above embodiments, upsampling the feature image can improve the image resolution, which is beneficial for more accurately determining the edge pixel positions of lane lines, pedestrians or other objects in the subsequent image recognition process, thereby improving the detection accuracy of the scene recognition model.

[0043] In an exemplary embodiment, in step S204 above, the feature fusion image obtained by the second attention module fusing the encoded image and the decoded image can be obtained in the following manner: the encoded image and the decoded image are input into the second attention module; the decoded image is subjected to feature extraction by the third convolution kernel in the second attention module to obtain local features of the decoded image; and the decoded image is subjected to feature extraction by the fourth convolution kernel in the second attention module to obtain global features of the decoded image; global average pooling is performed on the local features, the global features, and the encoded image to obtain the feature fusion image.

[0044] In an exemplary embodiment, in step S204 above, the scene recognition result obtained by the plurality of output modules recognizing the feature fusion image can be obtained by: inputting the feature fusion image into the output module to obtain a plurality of recognition results, wherein the output module includes a plurality of output sub-modules, each output sub-module corresponding to a scene recognition task, the scene recognition task including at least: lane line recognition task, drivable area recognition task, and target detection recognition task; determining the plurality of recognition results as the scene recognition result, wherein the plurality of recognition results including at least lane line recognition result, drivable area recognition result, and target detection recognition result.

[0045] In an exemplary embodiment, the scene recognition model in step S204 above can be trained as follows: a training image set and a test image set are obtained from different types of road scene images provided in an image library, wherein the road scene images in the image library are all labeled with image recognition results; the initial model is trained using the training image set as input samples and the image recognition results as output samples; the initial model is tested using the test image set, and training is stopped when the image recognition accuracy of the initial model is greater than a preset threshold, thereby obtaining the scene recognition model.

[0046] Optionally, in the above embodiments, the different types of road scene images provided in the image library include scene images for target detection and recognition, scene images for lane line recognition, and scene images for drivable area recognition. When configuring the training set and test set, 80% of the images can be used for training and 20% of the images can be used for testing.

[0047] Optionally, in the above embodiments, the training image set can also be enhanced and diversified to improve the model's generalization ability and robustness.

[0048] Obviously, the embodiments described above are only some embodiments of this application, and not all embodiments. To better understand the method for determining the road scenario described above, the process is explained below with reference to embodiments, but this is not intended to limit the technical solutions of the embodiments of this application. Specifically:

[0049] In an optional embodiment, Figure 3 This is a schematic diagram of the scene recognition model according to an embodiment of this application. The following is in conjunction with... Figure 3 The method for determining the road scene in this application is explained. Specifically, as follows... Figure 3 As shown, the input image (equivalent to a road scene image) is input to the encoder (equivalent to the encoder module). The encoder can use an ESPNet network to downsample the road scene image to obtain an encoded image. This encoded image is then input to the GCNet (Global Context Network) attention module (equivalent to the first attention module). The GCNet network can extract image feature information from the low-resolution feature map. Therefore, after dimensionality reduction, global feature learning can be performed on the image. After global feature learning is completed, the image is then upsampled to accelerate feature learning and obtain a feature image. This feature image is then input to the decoder (equivalent to the decoder module). The decoder module upsamples the feature image to increase its resolution, obtaining a decoded image. This is beneficial for more accurately determining the lane line edge pixel positions in subsequent image recognition processes, thereby improving the detection accuracy of the scene recognition model. Finally, the decoded image and the encoded image are input together into a high-resolution context attention module (equivalent to the second attention module) to obtain the lane line detection result. Figure 3 The process of recognizing drivable areas and targets is similar to that of lane line recognition, and will not be repeated here. It should be noted that in the process of recognizing drivable areas, point regression can be used instead of the traditional pixel segmentation method to speed up the recognition process.

[0050] In an optional embodiment, Figure 4 This is a schematic diagram of the structure of the first attention module of the scene recognition model according to an embodiment of this application, specifically as follows: Figure 4As shown, the GCNet attention module (equivalent to the first attention module) takes a C×H×W image as input. After dimensionality reduction, a C×1×1 image is obtained. Global information is learned using 1x1 convolution and the Softmax function (equivalent to a non-linear activation function). Optionally, the image channel information can also be learned using the non-linear activation function LayerNorm.ReLU. After learning, the image is then upscaled to restore the C×H×W image format.

[0051] In an optional embodiment, Figure 5 This is a schematic diagram of the structure of the second attention module of the scene recognition model according to an embodiment of this application, specifically as follows: Figure 5 As shown: The decoded image from the decoder module and the encoded image from the encoder are simultaneously input into the high-resolution attention module (equivalent to a second attention module). First, a standard 3x3 convolution is used to obtain local contextual information. Then, a dilated 3x3 convolution is used to increase the receptive field of the features, thereby obtaining global contextual information. The standard convolution and dilated convolution also apply Batch Normalization (BN) and the ReLU activation function to improve model performance. The outputs of the standard convolution and dilated convolution are fused through pooling to obtain global information. Next, the extracted global information is combined with the encoded image output from the encoder module, and after global average pooling, a feature fusion image is obtained. This feature fusion image is then input into the recognition layer to obtain the recognition result.

[0052] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods of the various embodiments of this application.

[0053] This embodiment also provides a road scene determination device, which is used to implement the above embodiments and preferred embodiments, and will not be repeated as already described. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0054] Figure 6This is a structural block diagram of a road scene determination device according to an embodiment of this application. The device includes:

[0055] Image acquisition module 62 is used to acquire images of the road scene where the vehicle is located using an image acquisition device, and obtain road scene images;

[0056] The scene recognition module 64 is used to input the road scene image into a scene recognition model and obtain the scene recognition result output by the scene recognition model. The scene recognition model includes: an encoder module, a first attention module, a decoder module, a second attention module, and an output module. The encoder module is used to acquire an encoded image obtained by encoding the road scene image. The first attention module is used to acquire a feature image obtained by extracting features from the encoded image. The decoder module is used to acquire a decoded image obtained by decoding the feature image. The second attention module is used to fuse the encoded image and the decoded image to obtain a feature fusion image. The output module is used to recognize the feature fusion image to obtain the scene recognition result.

[0057] The road scene determination device according to the embodiments of this application can acquire road scene images of the vehicle's road environment, and then perform encoding, feature extraction, decoding, and image fusion on the road scene images using a scene recognition model to obtain scene recognition results. This reduces the computational load of the image recognition process while improving the accuracy of scene image recognition, thereby accelerating the scene image recognition efficiency and improving the safety and response speed of autonomous driving. It solves the problem of how to improve the vehicle's efficiency in recognizing road scenes, thus achieving the effect of improving the vehicle's efficiency in recognizing road scenes.

[0058] In an exemplary embodiment, the scene recognition module 64 is further configured to: normalize the road scene image according to a preset format, input the normalized road scene image into the encoder module; and downsample the normalized road scene image through the encoder module to obtain the encoded image, wherein the resolution of the encoded image is smaller than the resolution of the road scene image, and the encoded image has image features with different sampling sizes.

[0059] In an exemplary embodiment, the scene recognition module 64 is further configured to: input the encoded image into the first attention module; perform dimensionality reduction processing on the feature dimension of the encoded image using a first convolutional kernel in the first attention module, wherein the feature dimension of the encoded image after dimensionality reduction is smaller than the feature dimension of the encoded image before dimensionality reduction; perform global feature learning on the encoded image after dimensionality reduction based on the nonlinear activation function of the first attention module to obtain an initial feature image; and perform dimensionality increase processing on the feature dimension of the initial feature image using a second convolutional kernel in the first attention module to obtain the feature image, wherein the feature dimension of the feature image is larger than the feature dimension of the initial feature image.

[0060] In an exemplary embodiment, the scene recognition module 64 is further configured to: input the feature image to the decoder module, and upsample the feature image through the decoder module to obtain the decoded image, wherein the resolution of the decoded image is greater than the resolution of the feature image.

[0061] In an exemplary embodiment, the scene recognition module 64 is further configured to: input the encoded image and the decoded image into the second attention module; extract features from the decoded image using a third convolutional kernel in the second attention module to obtain local features of the decoded image; and extract features from the decoded image using a fourth convolutional kernel in the second attention module to obtain global features of the decoded image; and perform global average pooling processing on the local features, the global features, and the encoded image to obtain the feature fusion image.

[0062] In an exemplary embodiment, the scene recognition module 64 is further configured to: input the feature fusion image into the output module to obtain multiple recognition results, wherein the output module includes multiple output sub-modules, each output sub-module corresponding to a scene recognition task, the scene recognition task including at least: lane line recognition task, drivable area recognition task, and target detection recognition task; and determine the multiple recognition results as the scene recognition result, wherein the multiple recognition results include at least lane line recognition result, drivable area recognition result, and target detection recognition result.

[0063] In an exemplary embodiment, the scene recognition module 64 is further configured to: obtain a training image set and a test image set from different types of road scene images provided in an image library, wherein the road scene images in the image library are all labeled with image recognition results; train an initial model using the training image set as input samples and the image recognition results as output samples; test the initial model using the test image set, and stop training when the image recognition accuracy of the initial model is greater than a preset threshold, thereby obtaining the scene recognition model.

[0064] Embodiments of this application also provide a storage medium including a stored program, wherein the program executes any of the above embodiments when it is run.

[0065] Optionally, in this embodiment, the storage medium may be configured to store a computer program for performing the following steps:

[0066] S1, Use an image acquisition device to acquire images of the road environment where the vehicle is located, and obtain road scene images;

[0067] S2, input the road scene image into the scene recognition model to obtain the scene recognition result output by the scene recognition model. The scene recognition model includes: an encoder module, a first attention module, a decoder module, a second attention module, and an output module. The encoder module is used to acquire an encoded image obtained by encoding the road scene image. The first attention module is used to acquire a feature image obtained by extracting features from the encoded image. The decoder module is used to acquire a decoded image obtained by decoding the feature image. The second attention module is used to fuse the encoded image and the decoded image to obtain a feature fusion image. The output module is used to recognize the feature fusion image to obtain the scene recognition result.

[0068] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0069] Specific examples in this embodiment can be found in the examples described in the above embodiments and exemplary implementations, and will not be repeated here.

[0070] Embodiments of this application also provide an electronic device including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the steps in any of the above method embodiments.

[0071] Optionally, in this embodiment, the processor can be configured to perform the following steps via a computer program:

[0072] S1, Use an image acquisition device to acquire images of the road environment where the vehicle is located, and obtain road scene images;

[0073] S2, input the road scene image into the scene recognition model to obtain the scene recognition result output by the scene recognition model. The scene recognition model includes: an encoder module, a first attention module, a decoder module, a second attention module, and an output module. The encoder module is used to acquire an encoded image obtained by encoding the road scene image. The first attention module is used to acquire a feature image obtained by extracting features from the encoded image. The decoder module is used to acquire a decoded image obtained by decoding the feature image. The second attention module is used to fuse the encoded image and the decoded image to obtain a feature fusion image. The output module is used to recognize the feature fusion image to obtain the scene recognition result.

[0074] In one exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor and the input / output device is connected to the processor.

[0075] Embodiments of this application also provide a computer program product, including a non-volatile computer-readable storage medium storing the computer program product, wherein the computer program, when executed by a processor, implements the steps of the methods described in various embodiments of this application.

[0076] Optionally, in this embodiment, the computer program described above can be configured to perform the following steps when executed by the processor:

[0077] S1, Use an image acquisition device to acquire images of the road environment where the vehicle is located, and obtain road scene images;

[0078] S2, input the road scene image into the scene recognition model to obtain the scene recognition result output by the scene recognition model. The scene recognition model includes: an encoder module, a first attention module, a decoder module, a second attention module, and an output module. The encoder module is used to acquire an encoded image obtained by encoding the road scene image. The first attention module is used to acquire a feature image obtained by extracting features from the encoded image. The decoder module is used to acquire a decoded image obtained by decoding the feature image. The second attention module is used to fuse the encoded image and the decoded image to obtain a feature fusion image. The output module is used to recognize the feature fusion image to obtain the scene recognition result.

[0079] Specific examples in this embodiment can be found in the examples described in the above embodiments and exemplary implementations, and will not be repeated here.

[0080] Obviously, those skilled in the art should understand that the modules or steps of this application described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. They can be implemented using computer-executable program code, and thus can be stored in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those presented here, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, this application is not limited to any particular combination of hardware and software.

[0081] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the principles of this application should be included within the protection scope of this application.

Claims

1. A method for determining a road scene, characterized in that, include: The image acquisition device is used to acquire images of the road environment in which the vehicle is located, and thus obtain road scene images; The road scene image is input into a scene recognition model to obtain a scene recognition result output by the scene recognition model. The scene recognition model includes an encoder module, a first attention module, a decoder module, a second attention module, and an output module. The encoder module is used to acquire an encoded image obtained by encoding the road scene image. The first attention module is used to acquire a feature image obtained by extracting features from the encoded image. The decoder module is used to acquire a decoded image obtained by decoding the feature image. The second attention module is used to fuse the encoded image and the decoded image to obtain a feature fusion image. The output module is used to recognize the feature fusion image to obtain the scene recognition result. The feature image obtained by the first attention module performing feature extraction on the encoded image is acquired in the following manner: The encoded image is input into the first attention module, and the feature dimension of the encoded image is reduced by the first convolution kernel in the first attention module. The feature dimension of the encoded image after the dimensionality reduction is smaller than the feature dimension of the encoded image before the dimensionality reduction. Global feature learning is performed on the dimensionality-reduced encoded image based on the nonlinear activation function of the first attention module to obtain an initial feature image; The feature dimension of the initial feature image is increased by using the second convolution kernel in the first attention module to obtain the feature image, wherein the feature dimension of the feature image is greater than the feature dimension of the initial feature image.

2. The method according to claim 1, characterized in that, The encoded image obtained by the encoder module encoding the road scene image is obtained in the following way: The road scene image is normalized according to a preset format, and the normalized road scene image is input into the encoder module. The encoder module downsamples the normalized road scene image to obtain the encoded image, wherein the resolution of the encoded image is smaller than the resolution of the road scene image.

3. The method according to claim 1, characterized in that, The decoded image obtained by the decoder module decoding the feature image is obtained in the following way: The feature image is input to the decoder module, and the decoder module upsamples the feature image to obtain the decoded image, wherein the resolution of the decoded image is greater than the resolution of the feature image.

4. The method according to claim 1, characterized in that, The feature fusion image, obtained by fusing the encoded image and the decoded image using the second attention module, is acquired in the following manner: The encoded image and the decoded image are input into the second attention module. The third convolution kernel in the second attention module is used to extract features from the decoded image to obtain local features of the decoded image. The fourth convolution kernel in the second attention module is used to extract features from the decoded image to obtain global features of the decoded image. The local features, the global features, and the encoded image are subjected to global average pooling to obtain the feature fusion image.

5. The method according to claim 1, characterized in that, The scene recognition result obtained by recognizing the feature fusion image from multiple output modules is acquired in the following way: The feature fusion image is input into the output module to obtain multiple recognition results. The output module includes multiple output sub-modules, each of which corresponds to a scene recognition task. The scene recognition task includes at least: lane line recognition task, drivable area recognition task, and target detection recognition task. The plurality of recognition results are determined as the scene recognition results, wherein the plurality of recognition results include at least lane line recognition results, drivable area recognition results, and target detection recognition results.

6. The method according to claim 1, characterized in that, The scene recognition model is trained using the following method: Training and test image sets are obtained from different types of road scene images provided in the image library, wherein the road scene images in the image library are all labeled with image recognition results; Using the training image set as input samples and the image recognition results as output samples, the initial model is trained. The initial model is tested using the test image set. Training is stopped when the image recognition accuracy of the initial model is greater than a preset threshold, thus obtaining the scene recognition model.

7. A device for determining a road scene, characterized in that, include: The image acquisition module is used to acquire images of the road environment where the vehicle is located using an image acquisition device, and obtain road scene images; A scene recognition module is used to input the road scene image into a scene recognition model and obtain the scene recognition result output by the scene recognition model. The scene recognition model includes an encoder module, a first attention module, a decoder module, a second attention module, and an output module. The encoder module is used to acquire an encoded image obtained by encoding the road scene image. The first attention module is used to acquire a feature image obtained by extracting features from the encoded image. The decoder module is used to acquire a decoded image obtained by decoding the feature image. The second attention module is used to fuse the encoded image and the decoded image to obtain a feature fusion image. The output module is used to recognize the feature fusion image to obtain the scene recognition result. The scene recognition module is further configured to input the encoded image into the first attention module, perform dimensionality reduction processing on the feature dimension of the encoded image using the first convolution kernel in the first attention module, wherein the feature dimension of the encoded image after dimensionality reduction is smaller than the feature dimension of the encoded image before dimensionality reduction; perform global feature learning on the dimensionality-reduced encoded image based on the nonlinear activation function of the first attention module to obtain an initial feature image; and perform dimensionality increase processing on the feature dimension of the initial feature image using the second convolution kernel in the first attention module to obtain the feature image, wherein the feature dimension of the feature image is larger than the feature dimension of the initial feature image.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored program, wherein the program, when executed, performs the method according to any one of claims 1 to 6.

9. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to execute the method described in any one of claims 1 to 6 through the computer program.

Images