Road obstacle recognition method and device

By using the improved ShuffleNetV2 architecture and the SkipNet model's jump network decision-making, the computation path is dynamically adjusted, solving the problems of high resource requirements and long processing time in road obstacle recognition, and achieving efficient and accurate obstacle recognition.

CN121147876BActive Publication Date: 2026-11-17CHINA MOBILE M2M +1
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202411740549.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-11-29
Publication Date
2026-11-17
Estimated Expiration
2044-11-29

AI Technical Summary

Technical Problem

Existing technologies for road obstacle recognition require high computational resources and have long processing times, making it difficult to achieve fast and efficient recognition, especially in resource-constrained environments.

Method used

We adopt an improved ShuffleNetV2 architecture combined with the SkipNet model and an extreme random tree gating network. Through the skip network decision mechanism, we dynamically adjust the computation path and decide whether to perform deep feature extraction based on image features, thereby optimizing resource utilization.

Benefits of technology

It improves the efficiency and accuracy of road obstacle recognition, optimizes resource utilization, and is particularly suitable for application scenarios with limited resources and high real-time requirements.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN121147876B_ABST
    Figure CN121147876B_ABST
Patent Text Reader

Abstract

This disclosure relates to a method and apparatus for road obstacle recognition. The method includes: acquiring road image data; inputting the road image data into a road obstacle recognition model to obtain the recognition result of obstacles in the road; wherein the road obstacle recognition model includes a first stage, a second stage, a third stage, and an output layer; the first and second stages are used to extract primary features from the road image data, and the third stage is used to extract high-level features from the road image data; wherein a skip network is provided between the second and third stages, the skip network is used to generate feature vectors based on the primary features, and to determine whether to skip the third stage based on the feature vectors; the output layer is used to output the recognition result of obstacles in the road based on the feature vectors or high-level features. This decision made by the skip network increases the flexibility and adaptability of the recognition, and can optimize processing according to real-time data conditions, optimizing resource utilization efficiency while ensuring recognition accuracy.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, and in particular to a method and apparatus for road obstacle recognition. Background Technology

[0002] With the continuous development of technology, automobiles and other means of transportation are becoming increasingly intelligent and have largely become an indispensable part of people's lives. However, with the increase in vehicles on the road, traffic pressure is also increasing, and traffic accidents are becoming more frequent. Especially on highways, if obstacles in the road are not cleared in time, accidents may occur.

[0003] Related technologies for identifying obstacles on roads primarily rely on various image processing and pattern recognition techniques for detection and recognition. For example, convolutional neural network architectures are used to identify road obstacles. While these technologies generally perform well in data processing, they typically require significant computational resources and processing time. If resources are limited or rapid processing is required, they may not achieve satisfactory recognition results. Summary of the Invention

[0004] This disclosure provides a method and apparatus for road obstacle recognition.

[0005] According to a first aspect of this disclosure, a road obstacle identification method is provided, the method comprising:

[0006] Acquire road image data;

[0007] The road image data is input into a road obstacle recognition model to obtain the recognition result of obstacles in the road; wherein, the road obstacle recognition model includes a first stage, a second stage, a third stage, and an output layer; the first stage and the second stage are used to extract primary features of the road image data, and the third stage is used to extract high-level features of the road image data; wherein, a skip network is further provided between the second stage and the third stage, the skip network is used to generate feature vectors based on the primary features, and to determine whether to skip the third stage based on the feature vectors; the output layer is used to output the recognition result of obstacles in the road based on the feature vectors or the high-level features.

[0008] According to a second aspect of this disclosure, a road obstacle recognition device is provided, the device comprising:

[0009] The data acquisition module is used to acquire road image data;

[0010] An obstacle recognition module is used to input the road image data into a road obstacle recognition model to obtain the recognition result of obstacles in the road. The road obstacle recognition model includes a first stage, a second stage, a third stage, and an output layer. The first and second stages are used to extract primary features from the road image data, and the third stage is used to extract high-level features from the road image data. A skip network is also provided between the second and third stages. The skip network is used to generate feature vectors based on the primary features and to determine whether to skip the third stage based on the feature vectors. The output layer is used to output the recognition result of obstacles in the road based on the feature vectors or the high-level features.

[0011] According to a third aspect of this disclosure, an electronic device is provided. The electronic device includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the program to implement the method described above.

[0012] According to a fourth aspect of this disclosure, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the methods described above.

[0013] According to a fifth aspect of this disclosure, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the methods described above.

[0014] The road obstacle recognition method and apparatus provided in this disclosure acquire road image data and input it into a road obstacle recognition model to obtain the recognition result of obstacles in the road. The road obstacle recognition model includes a first stage, a second stage, a third stage, and an output layer. The first and second stages are used to extract primary features from the road image data, and the third stage is used to extract high-level features from the road image data. A skip network is provided between the second and third stages. The skip network generates feature vectors based on the primary features and determines whether to skip the third stage based on the feature vectors. The output layer outputs the recognition result of obstacles in the road based on the feature vectors or high-level features. This skip network's determination of whether further processing through the third stage is needed increases the flexibility and adaptability of the recognition process, allowing for optimization based on real-time data conditions, and improving resource utilization efficiency while maintaining recognition accuracy. Attached Figure Description

[0015] Further details, features, and advantages of this disclosure are disclosed in the following description of exemplary embodiments in conjunction with the accompanying drawings, in which:

[0016] Figure 1A schematic diagram of a road obstacle recognition process provided for an exemplary embodiment of this disclosure;

[0017] Figure 2 A schematic diagram of a road obstacle recognition process provided for another exemplary embodiment of this disclosure;

[0018] Figure 3 A flowchart illustrating a road obstacle recognition method provided in an exemplary embodiment of this disclosure;

[0019] Figure 4 A schematic block diagram of the functional modules of a road obstacle recognition device provided in an exemplary embodiment of this disclosure;

[0020] Figure 5 A structural block diagram of an electronic device provided as an exemplary embodiment of this disclosure;

[0021] Figure 6 A block diagram of a computer system provided for an exemplary embodiment of this disclosure. Detailed Implementation

[0022] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0023] It should be understood that the steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.

[0024] The term "comprising" and its variations as used herein are open-ended, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below. It should be noted that the concepts of "first", "second", etc., used in this disclosure are only used to distinguish different devices, modules, or units, and are not intended to limit the order of functions performed by these devices, modules, or units or their interdependencies.

[0025] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0026] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0027] It is understood that before using the technical solutions disclosed in the various embodiments of this disclosure, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this disclosure in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained.

[0028] For example, upon receiving a user's active request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the software or hardware, such as the electronic device, application, server, or storage medium performing the operations of this disclosed technical solution, based on the prompt message.

[0029] As an optional but non-limiting implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device. It is understood that the above notification and user authorization process is merely illustrative and does not constitute a limitation on the implementation of this disclosure; other methods that comply with relevant laws and regulations may also be applied to the implementation of this disclosure.

[0030] In related technologies, such as using traditional CNN (Convolutional Neural Networks) architectures, a large amount of computing resources are often required to process complex image analysis tasks. In resource-constrained environments, such as mobile devices or in-vehicle systems, this may lead to slow system operation and affect real-time performance.

[0031] For example, related technologies utilize the ShuffleNet architecture for image processing and obstacle recognition. ShuffleNet is a lightweight CNN architecture designed for efficient operation on mobile devices, reducing computational costs through point convolutions and channel shuffling. However, the standard ShuffleNet architecture lacks an integrated advanced decision-making mechanism to dynamically adjust the processing flow, thus limiting its real-time performance and resource optimization. For instance, implementations in related technologies typically do not include dynamic decision support systems; they routinely process each frame of image without determining whether in-depth analysis is needed. This approach can lead to unnecessary resource waste and response delays when dealing with complex or variable road environments.

[0032] Therefore, although the ShuffleNet architecture optimizes computational efficiency, the standard configuration may still not meet the requirements for low power consumption or fast processing when dealing with highly complex or ever-changing traffic environments.

[0033] As a variant of the ShuffleNet architecture, ShuffleNetV2 processes all input images through a fixed network flow, ignoring the complexity of the input image content or the differences in actual requirements. This fixed processing flow leads to a problem: even when faced with simple images, the system performs all computational steps. As a result, when processing simple or low-complexity scenes, resource utilization is low, unnecessary computational burden is added, excessive energy is consumed, and processing delays may occur, which is particularly pronounced on resource-constrained devices.

[0034] Furthermore, the lack of a dynamic adjustment mechanism prevents ShuffleNetV2 from optimizing its network structure and computational resource allocation based on the actual image content. In real-time image processing and fast-response scenarios, this fixed network structure may lead to insufficient processing speed, especially when dealing with dynamically changing environments or sudden events.

[0035] To address the aforementioned technical problems, the embodiments of this disclosure, through an improved ShuffleNetV2 architecture and an extreme random tree gating network, not only enhance image processing efficiency but also significantly optimize resource utilization through intelligent decision-making, making them particularly suitable for real-time and resource-constrained application scenarios. By providing a more flexible and economical solution, these embodiments can significantly improve the real-time performance and accuracy of the system, effectively overcoming the shortcomings of related technologies.

[0036] This disclosure effectively solves the aforementioned technical problems by introducing the dynamic execution strategy of the SkipNet model and an Extra-Trees gated network. The embodiment can determine whether to execute certain parts of the improved network based on real-time analyzed image features. This allows the system to automatically skip unnecessary computational layers in simple scenarios, significantly reducing resource consumption and improving processing efficiency. For complex scenarios, the system can execute all layers to ensure accurate obstacle recognition.

[0037] Furthermore, through dynamic gating decisions, embodiments of this disclosure optimize the allocation of computing resources, adjusting the network activity portion according to the specific needs of each scenario. This optimization not only reduces energy consumption in simple scenarios but also ensures sufficient computing resources to process complex or hazardous images.

[0038] Therefore, the embodiments disclosed herein can not only significantly improve the efficiency of road obstacle recognition, but also optimize the performance of the system in various operating environments. They are particularly suitable for application scenarios with limited resources or extremely high real-time requirements, and can solve problems such as resource utilization efficiency and excessive network entity load in related technologies.

[0039] Specifically, this disclosure provides a road obstacle recognition method based on a modified ShuffleNetV2 architecture, which incorporates the gating concept of the SkipNet model and uses an extreme random tree as the gating decision model. Figure 1 As shown, Figure 1 The schematic diagram of the road obstacle recognition process provided in this embodiment of the disclosure may specifically include the following process:

[0040] (1) Data input.

[0041] In this embodiment, image data of the current road can be collected using cameras on vehicles or surveillance cameras on the road, or by setting up surveillance cameras at key locations on the road and continuously capturing video streams of road conditions to obtain the collected image data. The collected image data is then preprocessed and used as input data.

[0042] (2) Primary feature processing.

[0043] In the embodiment, in the initial feature processing stage, Stage 1 and Stage 2 of ShuffleNetV2 are mainly used as the main process.

[0044] The preprocessed image data needs to go through Stage 1, which includes an initial convolutional layer and a max pooling layer. The initial convolutional layer uses a 3x3 convolutional kernel and then sets the stride to 2. This is followed by the max pooling layer, which also uses the same pooling kernel and stride settings.

[0045] In this embodiment, the image data enters the basic feature extraction stage, namely Stage 2. This stage contains four ShuffleNet units, each performing channel splitting and channel shuffling operations. The depthwise separable convolutions in Stage 2 further reduce computational costs while maintaining or improving feature quality. The first unit of Stage 2 also employs a convolution operation with a stride of 2 to further reduce the size of the feature map.

[0046] It should be noted that the main function of Stage 1 and Stage 2 in the ShuffleNetV2 implementation is to build an efficient network structure to reduce computational complexity while maintaining high accuracy. Specifically, Stage 1 primarily reduces computational cost through channel splitting. Specifically, Stage 1 first divides the input feature map into two parts: one part undergoes channel grouping, while the other part remains unchanged. In this way, ShuffleNetV2 can reduce computational cost and improve model efficiency while maintaining feature reuse.

[0047] Stage 2's main function is to achieve more complex feature extraction by stacking multiple ShuffleNetV2 blocks. Each ShuffleNetV2 block includes operations such as channel grouping, depthwise separable convolution, feature rearrangement, and channel concatenation. These operations work together to achieve feature rearrangement and information exchange between channels, further enhancing the model's expressive power.

[0048] (3) Skip-layer network decision-making.

[0049] In ShuffleNetV2, the transition point from Stage 2 to Stage 3 is determined by setting a jump point. This jump point involves data feature extraction and dynamic decision-making for the jump.

[0050] In this embodiment, at the jump point, the network further processes the data processed in the previous step, extracting relevant features again. These features represent the state of the input data after passing through the current network layer. For example, these features can be processed by a global average pooling layer to reduce the complexity and computational requirements of subsequent processing.

[0051] The extracted features are then fed into the trained Extremely Random Tree model, which outputs a decision indicating whether to skip some subsequent layers. If the decision is made to skip, the data is directly passed to the final output layer, bypassing the intermediate ShuffleNetV2 Stage 3 processing; if the decision is not to skip, the data continues through Stage 3 processing and is then passed to the output layer.

[0052] This disclosed embodiment combines the techniques of ShuffleNetV2 and SkipNet, selecting a skip network at the transition point between Stage 2 and Stage 3 to set up key decision nodes. The core purpose of this design is to optimize network performance and improve resource utilization through intelligent decision-making, especially when dealing with complex and resource-sensitive image recognition tasks.

[0053] Stage 2 of ShuffleNetV2 is primarily responsible for basic feature extraction. Through a series of convolutions and channel shuffling operations, preliminary features have been extracted from the input image. These operations mainly aim to capture low- and mid-level visual features in the image, such as edges and textures, which are crucial for subsequent, more complex image understanding. By the end of Stage 2, a set of more complex feature maps has been formed, but in-depth semantic analysis has not yet been performed.

[0054] In traditional deep learning models, all input data passes through each stage of the network unconditionally. However, this approach often leads to a waste of computational resources, especially when the features are sufficient for effective decision-making. To address this issue, this implementation introduces SkipNet's gating mechanism at the transition point from Stage 2 to Stage 3. The skip network evaluates whether the feature map obtained from Stage 2 is sufficient and whether it is necessary to proceed to Stage 3 for deeper feature processing. This evaluation is based on the output of the gating network, i.e., the skip decision, which determines whether to proceed to Stage 3.

[0055] This design of the embodiment allows the model to skip certain computationally intensive network layers when necessary, thereby saving resources and reducing response time. Furthermore, this strategy improves the network's adaptability and flexibility under different operating conditions and environments, enabling the model to operate efficiently on resource-constrained devices and ensuring high practicality and scalability.

[0056] In this embodiment, the skip network is designed to optimize the computational efficiency and resource utilization of the entire network, especially in resource-constrained environments. At this node, an intelligent gating mechanism is introduced, which makes operational decisions based on the feature vector after global average pooling and the extreme random tree model.

[0057] Specifically, the process begins with Global Average Pooling (GAP). The purpose of this operation is to transform the multi-channel feature map output from Stage 2 into a simplified form, thereby reducing the complexity and computational requirements of subsequent processing.

[0058] Next, this feature vector is fed into a gating model based on extreme random trees. This model uses the feature vector to determine whether to activate the complex feature extraction process of Stage 3.

[0059] Extremely random trees play a crucial role here because they can quickly learn from these features and make decisions; their decision function can be expressed as:

[0060] [y=Extra-Trees(x)] (1)

[0061] Here, x is the input feature vector, and y is a binary output indicating whether to skip Stage 3. This decision mechanism allows the network to selectively skip certain computational stages without sacrificing recognition accuracy, thereby achieving higher efficiency and lower resource consumption.

[0062] By combining global average pooling with extreme random trees, the implementation achieves a balance between high recognition performance and real-time response, making it particularly suitable for deployment in computationally limited environments and ensuring efficient and flexible obstacle recognition capabilities.

[0063] In the embodiments provided in this disclosure, the extreme random tree model can demonstrate excellent efficiency and robustness when processing high-dimensional data, and is particularly suitable for real-time and dynamic decision-making environments, such as the vision processing system of autonomous vehicles.

[0064] The core advantage of Extremely Random Trees (ERTs) lies in their fast construction speed and strong generalization ability to data. This is achieved by introducing two main randomization steps when constructing the decision tree: first, adopting a random strategy when selecting splitting features; and second, using random split points for each feature, rather than searching for the optimal split point. This construction method not only speeds up the training process but also increases the diversity of the model through this randomness, thereby reducing the risk of overfitting.

[0065] Another advantage of extreme random trees is their efficiency, because once the tree is built, decisions can be made with just a few comparisons and branches, making them ideal for real-time systems. Furthermore, extreme random trees can effectively handle nonlinear relationships and complex interaction effects, which is particularly important when processing image data, as there are often complex spatial relationships between image features.

[0066] In this embodiment of the disclosure, the feature vector after global average pooling provides compressed global image information. Extreme random trees use this information to quickly determine whether to enter Stage 3 for further feature analysis.

[0067] In the embodiments provided in this disclosure, the output of Stage 2 of ShuffleNetV2 is a multi-dimensional feature map with multiple channels. Global average pooling plays a crucial role in processing the output of Stage 2 of ShuffleNetV2. Its purpose is to simplify the spatial information of each channel into a single numerical value, which helps reduce the computational burden on the model in subsequent stages and improves processing speed. Global average pooling is achieved by averaging all pixel values ​​of each channel, effectively compressing the features of each channel into a scalar, thereby reducing the overall feature dimensionality.

[0068] Specifically, if each channel of the feature map is (M×N), the global average pooling operation will calculate the average of all pixel values ​​in each channel. This can be expressed by the following formula:

[0069]

[0070] Where f i ( j k) Let x represent the feature value of the k-th channel at position (i, j). k It is the average value of the k-th channel, forming a new feature vector that includes the average values ​​of all channels.

[0071] Global average pooling not only improves the efficiency of subsequent processing but also helps prevent overfitting by forcing the network to capture the most representative features rather than details at specific locations. Furthermore, global average pooling simplifies the network's output, reducing the complex mapping from high-dimensional features to the final decision layer, thus making the entire network structure more concise and efficient.

[0072] (4) Advanced feature processing.

[0073] In the embodiment, if the skip network executes Stage 3, it enters the advanced feature processing step. Stage 3 includes 8 ShuffleNet units, each of which uses depthwise separable convolutions and point convolutions to further refine and enhance features.

[0074] It's important to note that the main purpose of Stage 3 in ShuffleNetV2 is to further extract and fuse features to enhance the model's expressive power and accuracy. In Stage 3, ShuffleNetV2 further extracts and fuses features through multiple basic units (such as the Shuffle module). These modules enhance feature interactions and improve the model's expressive power through operations such as grouped convolution, channel shuffling, and element-wise addition.

[0075] (5) Output the results.

[0076] If the skip network decides not to execute Stage 3, it proceeds to the final result output stage.

[0077] In this embodiment, the output layer comprises four ShuffleNet units, which further enhance feature processing through channel shuffling and depthwise separable convolutions. At the end of the network, a global average pooling layer compresses the feature map of each channel into a single value, significantly reducing the number of parameters in the fully connected layers. A fully connected layer then maps the pooled features to the final category output, completing obstacle recognition.

[0078] In the embodiments provided in this disclosure, the extreme random tree plays a decisive role in the efficiency and performance optimization of the entire network. The extreme random tree determines whether to skip Stage 3 of ShuffleNetV2 by using the feature vectors from the output of Stage 2 of ShuffleNetV2, which have undergone global average pooling, thereby optimizing resource usage and reducing computation time. The following are the detailed steps for training the extreme random tree model.

[0079] (1) Generation of eigenvectors.

[0080] The example acquires a training dataset consisting of multiple images of roads. Feature vectors are generated based on the image data in the training dataset. Each image is processed using Stage 2 of ShuffleNetV2 to extract multi-channel feature maps. These feature maps are then subjected to global average pooling to generate feature vectors.

[0081] Global average pooling is achieved by calculating the average value of all elements in each channel, which can be achieved by the above formula (2). In this way, each image can be converted into a feature vector x of size M×N containing the average value of each channel.

[0082] (2) Construction of extreme random trees.

[0083] Training an extreme random tree is a process involving building multiple independent decision trees to capture complex structures and patterns from data. This approach is particularly well-suited for scenarios requiring rapid decision-making because it does not pursue the optimal split for each node, but rather increases the diversity and generalization ability of the tree by adding randomness. The training steps for extreme random trees are described in detail below.

[0084] The first step in training is selecting a subset of features. At the start of building each tree, a subset of features is randomly selected from all available features. This step is crucial for increasing the diversity between trees. The selection of which features will participate in tree construction is random, but it is usually based on a fixed proportion of all features. This approach ensures that each tree can be trained on a different subset of information, thereby increasing the overall robustness of the model.

[0085] The second step of training is to randomly select the split point. For each node in the tree, once the feature used for splitting is determined, the next step is to decide the split point on that feature. Unlike traditional decision trees that search for the optimal split point on each feature, extreme random trees randomly select a split point on each feature. This process does not require calculating statistics such as information gain or Gini impurity, thus greatly speeding up the tree construction. The specific splitting operation can be expressed by the following formula:

[0086] [Split(S,k,t) = (x,y)∈S:x k ≤t] (3)

[0087] The third step in training is node segmentation. Once the segmentation points are determined, the samples within a node are segmented based on whether they meet the segmentation condition (x). k The sample size (≤t) is divided into two parts, each corresponding to one of the tree's two child nodes. This process continues until a stopping condition is met. Common stopping conditions include: the number of samples in a node is less than a preset threshold, or all samples belong to the same class. These conditions help prevent overfitting and ensure that the tree does not grow too complex.

[0088] In this way, extreme random trees introduce more randomness at each decision node, enabling the model to be trained on multiple different data slices, thus improving overall predictive performance and robustness. At the same time, this randomness also means that the model has a higher tolerance for noise and outliers during training.

[0089] (3) Model training.

[0090] By repeating the above process, the extreme random tree model is trained on the training set until all trees are constructed. Each tree learns based on the generated feature vectors, eventually reaching a point where it can distinguish between situations with and without obstacles.

[0091] (4) Use and verification.

[0092] After training, the model can be used for new image data. For each new image, the same feature extraction and global average pooling process as during training is performed, and then the trained extreme random tree model is used to predict whether to skip Stage 3. This prediction can be based on the above formula (1), where y represents the model's predicted output, indicating whether Stage 3 needs to be executed.

[0093] This process ensures that the model maintains high accuracy while optimizing performance and resource usage by skipping unnecessary computations.

[0094] In conjunction with the above embodiments, the embodiments provided in this disclosure can be specifically applied to, for example, highway monitoring systems, using cameras to monitor road conditions in real time in order to detect and respond to various obstacles, such as debris flying in from the outside, items left behind by vehicles, road damage (such as road cracks or collapses), and traffic jams caused by traffic accidents.

[0095] First, the monitoring system relies on high-quality camera equipment capable of clearly capturing road images under various lighting and weather conditions. The cameras should have high resolution and a sufficient frame rate to capture fast-moving objects while ensuring image clarity, which is fundamental to the system's detection accuracy.

[0096] Secondly, the monitoring system requires a backend processing unit, which can be implemented using a server installed near the roadside or by utilizing cloud computing resources. These processing units need sufficient computing power to run the methods involved in the embodiments of this disclosure, processing the video streams transmitted from the cameras in real time.

[0097] Furthermore, the implementation of this system should ideally be coordinated with the local traffic management system to enable swift action when serious obstacles are detected. This includes integration with emergency response systems or direct links to traffic signal control systems to adjust traffic flow and alleviate traffic congestion in specific areas.

[0098] Specifically, such as Figure 2 As shown, Figure 2 This is a schematic diagram of a road obstacle recognition process provided in another embodiment of the present disclosure. Specifically, it may include the following steps:

[0099] (1) Data collection and preliminary processing.

[0100] Image data of the current road can be collected using cameras on vehicles or surveillance cameras on the road. For example, while a vehicle is in motion, a camera installed on the vehicle can collect real-time image data of the current road. Alternatively, surveillance cameras can be installed at key locations on the road and continuously capture video streams of road conditions to obtain the collected image data. These key locations typically include intersections, accident-prone areas, and other important traffic nodes to ensure that the system can comprehensively monitor and record various situations on the road.

[0101] Before being sent to the processing center, image data can undergo preprocessing to improve its usability. This includes automatically adjusting image sharpness to cope with different ambient lighting conditions; applying advanced denoising techniques to eliminate random noise in the image, which is especially important at night or in inclement weather conditions; and performing color correction to ensure the authenticity and consistency of image colors, thereby making subsequent feature recognition more accurate. These preprocessing operations ensure the quality of the input data and improve the system's sensitivity and accuracy in obstacle recognition.

[0102] After image preprocessing, including image data sharpening and denoising, and color correction, the preprocessed image data is converted into a format suitable for deep learning models. The video stream is then segmented into single frames or mini-batches, image sizes are adjusted to fit the model's input requirements, and image files are converted into specific data types and structures. After these processes, the data can be used for subsequent feature extraction and analysis.

[0103] (2) Feature extraction and jump network decision.

[0104] In this embodiment, the system can begin by receiving the pre-processed image data described above, which has been optimized to suit the needs of the deep learning model. Building upon this, Stage 2 of the modified ShuffleNetV2 architecture is used, focusing on efficiently extracting deep features from these images. This stage leverages its advanced network design, employing grouped convolutions and channel shuffling techniques to extract fundamental but crucial visual information such as edges, textures, and shapes from the images.

[0105] After feature extraction, the system employs global average pooling. The purpose of global average pooling is to reduce the dimensionality and complexity of the data while retaining key information. The feature vectors after global average pooling are then fed into a skip network. The core function of the skip network is to evaluate these feature vectors and make decisions based on an extreme random tree model. It determines whether to proceed to Stage 3 of the ShuffleNetV2 architecture for deeper feature analysis or to jump to the final stage. This decision is crucial for optimizing the system's computational efficiency and response speed.

[0106] This step concludes with the gating network making a decision and outputting whether to activate Stage 3. This decision ensures that the system maintains high recognition accuracy while maximizing resource utilization and processing speed.

[0107] (3) Advanced feature processing.

[0108] Advanced feature processing is the stage in the system where in-depth feature analysis is performed. Its main goal is to extract more complex and abstract features from images, laying a solid foundation for accurate obstacle recognition.

[0109] Whether this step begins depends on the decision of the skip network. When the skip network decides not to skip Stage 3, the image data is introduced into Stage 3, initiating a deeper feature extraction process. This signifies that the system needs to perform more detailed analysis to ensure the accuracy of the recognition results, especially in complex or ambiguous scenes.

[0110] The completion of this step signifies that the image data has been fully analyzed, and all key information has been captured by the system and is ready for decision output, such as the type and location of the identified obstacles and their potential impact on traffic.

[0111] (4) Output and response.

[0112] Output and response are the final stages of a road obstacle recognition system. The main task is to classify and identify obstacles based on the features extracted and processed in the previous stages, and then generate the necessary outputs and take corresponding response measures.

[0113] The data for this step comes from two sources: high-level features extracted from Stage 3 and feature data received from the jump network. This feature data includes all the necessary information about the obstacles, such as their type, size, location, and potential impact, providing a basis for the final decision.

[0114] This stage involves analyzing the feature data to determine the types of obstacles present in the image. This can range from simple objects like stones or discarded items on the road to more complex situations such as traffic jams or road damage caused by car accidents. The classification process relies on trained algorithms capable of accurately identifying various obstacles from the features and assessing their potential impact on traffic.

[0115] Once an obstacle is correctly identified, the system generates appropriate output and triggers response measures. This includes automatically sending alerts to traffic management centers, providing detailed information about the obstacle to facilitate swift action; activating road surface warning displays, such as electronic signs or variable information signs, to warn drivers of impending danger; or, in more urgent situations, directly interfaceing with emergency services systems to ensure rapid incident response and handling.

[0116] This disclosure is based on a modified ShuffleNetV2 architecture. Specific improvements have been made to the ShuffleNetV2 architecture, including optimizations to network layers and parameters, to enhance its ability to process highly complex image data with low resource consumption. Optimizations to channel splitting and shuffling operations make the network more suitable for real-time processing of large amounts of surveillance video data. Furthermore, the embodiment employs a skip network strategy, allowing the system to automatically adjust processing based on the complexity of current image features. This increases the system's flexibility and adaptability, enabling optimization of the processing path based on real-time data conditions, thereby maximizing resource utilization while ensuring recognition accuracy.

[0117] Therefore, this embodiment of the disclosure employs an improved ShuffleNetV2 architecture. Compared to traditional deep learning models, the improved ShuffleNetV2 maintains a lightweight design while significantly improving network computational efficiency through optimized channel splitting and shuffling mechanisms. This enables high-quality image processing and feature recognition with lower computational resource consumption in resource-constrained environments, resolving the contradiction between processing speed and resource consumption in related technologies.

[0118] Furthermore, this embodiment introduces extreme random trees as the decision model for the gating network. Because extreme random trees are fast in training and decision-making, they can selectively skip certain network layers based on real-time data analysis needs and the current network state, thereby optimizing overall computational efficiency. This strategy not only reduces unnecessary computational burden but also accelerates the system's response time, significantly improving the system's practicality and efficiency, especially when rapid identification and processing of complex traffic situations are required.

[0119] Based on the above embodiments, this disclosure first provides a road obstacle recognition method, such as... Figure 3 As shown, the method may include the following steps:

[0120] In step S310, road image data is acquired.

[0121] In this embodiment, image data of the current road can be collected using a camera on the vehicle or a surveillance camera on the road. For example, while the vehicle is in motion, image data of the current road can be collected in real time using a camera installed on the vehicle.

[0122] In this embodiment, surveillance cameras can also be installed at key locations on the road to continuously capture video streams of road conditions and obtain collected image data. These key locations typically include intersections, accident-prone areas, and other important traffic nodes to ensure that the system can comprehensively monitor and record various situations on the road.

[0123] In this embodiment, the acquired image data can also be preprocessed. For example, road image data can be obtained by acquiring road images of the target road in real time and preprocessing the road images. Preprocessing includes: sharpness enhancement, noise reduction, or color correction.

[0124] In this embodiment, the preprocessing process may include adjusting the image sharpness to cope with different ambient lighting conditions. Advanced denoising techniques may also be applied to eliminate random noise in the image, which is particularly important at night or in inclement weather conditions. Color correction may also be performed on the image to ensure color accuracy and consistency, thereby making subsequent feature recognition more accurate. These preprocessing operations ensure the quality of the input data and improve the system's sensitivity and accuracy in obstacle recognition.

[0125] In step S320, road image data is input into the road obstacle recognition model to obtain the recognition results of obstacles in the road.

[0126] The road obstacle recognition model includes a first stage, a second stage, a third stage, and an output layer. The first and second stages are used to extract primary features from road image data, and the third stage is used to extract high-level features from road image data. A skip network is set between the second and third stages. The skip network is used to generate feature vectors based on the primary features and to determine whether to skip the third stage based on the feature vectors. The output layer is used to output the recognition results of obstacles in the road based on the feature vectors or high-level features.

[0127] In this embodiment, the obstacle recognition model can be the improved ShuffleNetV2 described in the previous embodiment, with the first stage being Stage 1, the second stage being Stage 2, and the third stage being Stage 3. By setting up a skip network at the transition point between Stage 2 and Stage 3, and setting key decision nodes, the network performance can be optimized and resource utilization improved through the decisions of the skip network, especially when dealing with complex and resource-sensitive image recognition tasks.

[0128] The skip network evaluates whether the feature map obtained from Stage 2 is sufficient to obtain the recognition result. If so, it generates a feature vector from the primary features obtained from Stage 2 and transmits it to the output layer to output the recognition result of the obstacle in the road. Otherwise, it needs to go through Stage 3 to extract deeper high-level features and transmit them to the output layer, which then outputs the recognition result of the obstacle in the road.

[0129] In this embodiment, when generating feature vectors based on primary features and determining whether to skip the third stage based on the feature vectors, a skip network can be used to perform global average pooling on the primary features to obtain the feature vectors. The feature vectors are then input into the extreme random tree model in the skip network, and the output shows whether the third stage has been skipped.

[0130] In this embodiment, the global average pooling process can convert the multi-channel feature map output by Stage 2 into a simplified form, thereby reducing the complexity and computational requirements of subsequent processing.

[0131] The road obstacle recognition method provided in this disclosure acquires road image data and inputs it into a road obstacle recognition model to obtain the recognition result of obstacles in the road. The road obstacle recognition model includes a first stage, a second stage, a third stage, and an output layer. The first and second stages are used to extract primary features from the road image data, and the third stage is used to extract high-level features from the road image data. A skip network is provided between the second and third stages. The skip network generates feature vectors based on the primary features and determines whether to skip the third stage based on the feature vectors. The output layer outputs the recognition result of obstacles in the road based on the feature vectors or high-level features. This skip network's determination of whether further processing through the third stage is needed increases the flexibility and adaptability of the recognition process, allowing for optimization based on real-time data conditions, and improving resource utilization efficiency while maintaining recognition accuracy.

[0132] Based on the above embodiments, when the output layer outputs the identification results of obstacles in the road based on feature vectors or high-level features, if the output result skips the third stage, the output layer can be used to output the identification results of obstacles in the road based on feature vectors. Alternatively, if the output result does not skip the third stage, high-level features are extracted from the road image data based on feature vectors in the third stage, and the output layer can be used to output the identification results of obstacles in the road based on high-level features.

[0133] In the embodiment, the feature vector after global average pooling can provide compressed global image information. Based on this global image information, the extreme random tree can quickly determine whether it needs to enter Stage 3, that is, whether it needs to perform more in-depth feature analysis through Stage 3.

[0134] In this embodiment, when constructing the extreme random tree model in the skip network, training data can be obtained and the extreme random tree model can be constructed based on this training data. The training data includes feature vectors generated from road images of multiple roads. For details, please refer to the description of the extreme random tree model training process in the above embodiments; it will not be repeated here.

[0135] In the embodiments provided in this disclosure, the category information of obstacles in the road can be obtained based on the above identification results. Road warning information can then be generated based on this category information.

[0136] For example, effective measures can be taken promptly based on the category information of obstacles on the road. For instance, when the category information is "road collapse," a road repair warning can be generated in a timely manner to facilitate road repair. When the category information is "tire" or other road obstacles, an obstacle removal warning can be generated to facilitate timely removal of the obstacles and prevent traffic accidents.

[0137] By dividing each function into corresponding functional modules, this disclosure provides a road obstacle recognition device, which can be a server, a terminal, or a chip applied to a server. Figure 4 This is a schematic block diagram illustrating the functional modules of a road obstacle recognition device provided as an exemplary embodiment of this disclosure. Figure 4 As shown, the road obstacle recognition device includes:

[0138] Data acquisition module 10 is used to acquire road image data;

[0139] The obstacle recognition module 20 is used to input the road image data into a road obstacle recognition model to obtain the recognition result of obstacles in the road; wherein, the road obstacle recognition model includes a first stage, a second stage, a third stage, and an output layer; the first stage and the second stage are used to extract primary features of the road image data, and the third stage is used to extract high-level features of the road image data; wherein, a skip network is further provided between the second stage and the third stage, the skip network is used to generate feature vectors based on the primary features, and determine whether to skip the third stage based on the feature vectors; the output layer outputs the recognition result of obstacles in the road based on the feature vectors or the high-level features.

[0140] In another embodiment provided in this disclosure, the obstacle recognition module is specifically used for:

[0141] The primary features are processed by global average pooling through the skip network to obtain the feature vector;

[0142] The feature vector is input into the extreme random tree model in the skip network, and the output is the result of whether the third stage is skipped.

[0143] In another embodiment provided in this disclosure, the obstacle recognition module is further configured to:

[0144] If the output result is to skip the third stage, the output layer is used to output the identification result of obstacles in the road based on the feature vector;

[0145] If the output result does not skip the third stage, the third stage extracts high-level features from the road image data based on the feature vector, and the output layer is used to output the recognition result of obstacles in the road based on the high-level features.

[0146] In another embodiment provided in this disclosure, the apparatus further includes a training module, specifically used for:

[0147] Acquire training data, which includes feature vectors generated based on road images from multiple roads;

[0148] The extreme random tree model is constructed based on the training data.

[0149] In another embodiment provided in this disclosure, the data acquisition module is specifically used for:

[0150] Real-time acquisition of road images in the target road;

[0151] The road image is preprocessed to obtain the road image data; wherein the preprocessing includes: sharpness processing, noise reduction processing or color correction processing.

[0152] In yet another embodiment provided in this disclosure, the apparatus further includes:

[0153] The category information acquisition module is used to acquire category information of obstacles in the road based on the recognition results;

[0154] The early warning module is used to generate road warning information based on the category information.

[0155] The road obstacle recognition device provided in this disclosure acquires road image data and inputs it into a road obstacle recognition model to obtain the recognition result of obstacles in the road. The road obstacle recognition model includes a first stage, a second stage, a third stage, and an output layer. The first and second stages are used to extract primary features from the road image data, and the third stage is used to extract high-level features from the road image data. A skip network is provided between the second and third stages. The skip network generates feature vectors based on the primary features and determines whether to skip the third stage based on the feature vectors. The output layer outputs the recognition result of obstacles in the road based on the feature vectors or high-level features. This skip network's determination of whether further processing through the third stage is needed increases the flexibility and adaptability of the recognition process, allowing for optimized processing based on real-time data conditions, thus optimizing resource utilization efficiency while maintaining recognition accuracy.

[0156] This disclosure also provides an electronic device, including: at least one processor; a memory for storing processor-executable instructions; wherein the at least one processor is configured to execute the instructions to implement the methods disclosed in this disclosure.

[0157] Figure 5 This is a schematic diagram of the structure of an electronic device provided as an exemplary embodiment of this disclosure. For example... Figure 5 As shown, the electronic device 1800 includes at least one processor 1801 and a memory 1802 coupled to the processor 1801. The processor 1801 can perform the corresponding steps in the methods disclosed in the embodiments of this disclosure.

[0158] The processor 1801 described above can also be called a central processing unit (CPU), which can be an integrated circuit chip with signal processing capabilities. Each step in the method disclosed in this embodiment can be implemented by the integrated logic circuitry in the processor 1801 or by software instructions. The processor 1801 can be a general-purpose processor, a digital signal processor (DSP), an ASIC (Application Specific Integrated Circuit), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this embodiment can be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software modules can be located in the memory 1802, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The processor 1801 reads information from the memory 1802 and, in conjunction with its hardware, completes the steps of the method described above.

[0159] Furthermore, various operations / processes according to this disclosure, implemented via software and / or firmware, can be transmitted from a storage medium or network to a computer system with a dedicated hardware architecture, such as... Figure 6 The computer system 1900 shown is equipped with the programs that constitute the software. When various programs are installed, the computer system is able to perform various functions, including those described above. Figure 6 A block diagram of a computer system provided for an exemplary embodiment of this disclosure.

[0160] Computer System 1900 is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Electronic devices can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0161] like Figure 6As shown, the computer system 1900 includes a computing unit 1901, which can perform various appropriate actions and processes based on a computer program stored in a read-only memory (ROM) 1902 or a computer program loaded from a storage unit 1908 into a random access memory (RAM) 1903. The RAM 1903 may also store various programs and data required for the operation of the computer system 1900. The computing unit 1901, ROM 1902, and RAM 1903 are interconnected via a bus 1904. An input / output (I / O) interface 1905 is also connected to the bus 1904.

[0162] Multiple components in computer system 1900 are connected to I / O interface 1905, including: input unit 1906, output unit 1907, storage unit 1908, and communication unit 1909. Input unit 1906 can be any type of device capable of inputting information into computer system 1900. Input unit 1906 can receive input digital or character information and generate key signal inputs related to user settings and / or function control of the electronic device. Output unit 1907 can be any type of device capable of presenting information and may include, but is not limited to, a monitor, speaker, video / audio output terminal, vibrator, and / or printer. Storage unit 1908 may include, but is not limited to, hard disks and optical disks. Communication unit 1909 allows computer system 1900 to exchange information / data with other devices via a network such as the Internet, and may include, but is not limited to, modems, network cards, infrared communication devices, wireless communication transceivers, and / or chipsets, such as Bluetooth™ devices, WiFi devices, WiMax devices, cellular communication devices, and / or the like.

[0163] The computing unit 1901 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 1901 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 1901 performs the various methods and processes described above. For example, in some embodiments, the methods disclosed in this disclosure can be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 1908. In some embodiments, part or all of the computer program can be loaded and / or installed on an electronic device via ROM 1902 and / or communication unit 1909. In some embodiments, the computing unit 1901 can be configured to perform the methods disclosed in this disclosure by any other suitable means (e.g., by means of firmware).

[0164] This disclosure also provides a computer-readable storage medium, wherein when the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is able to perform the methods disclosed in this disclosure.

[0165] The computer-readable storage medium in this disclosure can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. The aforementioned computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specifically, the aforementioned computer-readable storage medium may include electrical connections based on one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0166] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0167] This disclosure also provides a computer program product, including a computer program, wherein the computer program, when executed by a processor, implements the methods disclosed in the embodiments of this disclosure.

[0168] In embodiments of this disclosure, computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof. These programming languages ​​include, but are not limited to, object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network (including a local area network (LAN) or a wide area network (WAN)), or it can be connected to an external computer.

[0169] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0170] The modules, components, or units described in the embodiments of this disclosure can be implemented in software or hardware. The names of the modules, components, or units do not necessarily constitute a limitation on the module, component, or unit itself.

[0171] The functions described above in this document can be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary hardware logic components that can be used include: field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), and so on.

[0172] The above description is merely an embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0173] While specific embodiments of this disclosure have been described in detail by way of example, those skilled in the art should understand that the examples are for illustrative purposes only and not intended to limit the scope of this disclosure. Those skilled in the art should understand that modifications can be made to the above embodiments without departing from the scope and spirit of this disclosure. The scope of this disclosure is defined by the appended claims.

Claims

1. A road obstacle recognition method based on a modified ShuffleNetV2 architecture, characterized in that, The method includes: Acquire road image data; The road image data is input into a road obstacle recognition model to obtain the recognition results of obstacles in the road. The road obstacle recognition model includes Stage 1, Stage 2, Stage 3, and an output layer. Stage 1 and Stage 2 are used to extract primary features from the road image data, and Stage 3 is used to extract high-level features from the road image data. A skip network is also provided between Stage 2 and Stage 3. The skip network is used to generate feature vectors based on the primary features and to determine whether to skip Stage 3 based on the feature vectors. The output layer is used to output the recognition results of obstacles in the road based on the feature vectors or the high-level features. The step of generating a feature vector based on the primary features and determining whether to skip Stage 3 based on the feature vector includes: The primary features are processed by global average pooling through the skip network to obtain the feature vector; The feature vector is input into the extreme random tree model in the skip network, and the output is whether to skip Stage 3.

2. The method according to claim 1, characterized in that, The output layer is used to output the recognition result of obstacles in the road based on the feature vector or the high-level feature, including: If the output result is to skip Stage 3, the output layer is used to output the identification result of obstacles in the road based on the feature vector; Alternatively, if the output result is that Stage 3 is not skipped, high-level features in the road image data are extracted based on the feature vector through Stage 3, and the output layer is used to output the recognition result of obstacles in the road based on the high-level features.

3. The method according to claim 1, characterized in that, The method further includes: Acquire training data, which includes feature vectors generated based on road images from multiple roads; The extreme random tree model is constructed based on the training data.

4. The method according to claim 1, characterized in that, The acquisition of road image data includes: Real-time acquisition of road images in the target road; The road image is preprocessed to obtain the road image data.

5. The method according to claim 1, characterized in that, The method further includes: Based on the recognition results, obtain the category information of obstacles in the road; Based on the category information, road warning information is generated.

6. A road obstacle recognition device based on a modified ShuffleNetV2 architecture, characterized in that, The device includes: The data acquisition module is used to acquire road image data; An obstacle recognition module is used to input the road image data into a road obstacle recognition model to obtain the recognition results of obstacles in the road. The road obstacle recognition model includes Stage 1, Stage 2, Stage 3, and an output layer. Stage 1 and Stage 2 are used to extract primary features from the road image data, and Stage 3 is used to extract high-level features from the road image data. A skip network is also provided between Stage 2 and Stage 3. The skip network is used to generate feature vectors based on the primary features and to determine whether to skip Stage 3 based on the feature vectors. The output layer is used to output the recognition results of obstacles in the road based on the feature vectors or the high-level features. The step of generating a feature vector based on the primary features and determining whether to skip Stage 3 based on the feature vector includes: The primary features are processed by global average pooling through the skip network to obtain the feature vector; The feature vector is input into the extreme random tree model in the skip network, and the output is whether to skip Stage 3.

7. An electronic device, characterized in that, include: At least one processor; Memory for storing the at least one processor-executable instruction; The at least one processor is configured to execute the instructions to implement the method as described in any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, When the instructions in the computer-readable storage medium are executed by the processor of the electronic device, the electronic device is able to perform the method as described in any one of claims 1-5.

9. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method according to any one of claims 1-5.

Citation Information

Patent Citations

  • Learned forensic source system for identification of image capture device models and forensic similarity of digital images

    US20220198711A1

  • Method of combinatorial optimization using hybrid temporo-attentional branching, and related system and devices

    US20230376558A1