Road recognition method and device, and wearable electronic device

Abstract

The present application discloses a road identification method and device, and a wearable electronic device. The method includes: determining, from a plurality of road damage images, a first image showing that the road has been repaired and is qualified, a second image showing that the road has been repaired and is unqualified, and a third image showing that the road has not been repaired; performing feature extraction on the first image, the second image, and the third image, respectively, to obtain a plurality of first features, a plurality of second features, and a plurality of third features; combining the plurality of first features, the plurality of second features, and the plurality of third features, respectively; training a preset neural network model to obtain a trained target neural network model, and using the trained target neural network model to identify the road damage video to be identified that was collected from the target object, and determine the road repair result. The present application solves the technical problem of low efficiency in road acceptance and quality inspection caused by the need for manual identification of the road repair status in related technologies.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and more specifically, to a road recognition method and device, and a wearable electronic device. Background Art

[0002] The highway maintenance industry, especially the maintenance of national and provincial trunk roads and rural roads, is often faced with problems such as long maintenance mileage, lack of manpower, and insufficient funds. How to improve management and maintenance efficiency is an issue that the current maintenance industry urgently needs to solve.

[0003] However, the related technology requires manual identification of the maintenance status of the road, which results in low efficiency in road acceptance and quality inspection.

[0004] To address the above-mentioned problems, no effective solutions have been proposed so far. Summary of the Invention

[0005] The embodiments of the present application provide a road identification method and device, and a wearable electronic device to at least solve the technical problem of low efficiency in road acceptance and quality inspection caused by the need for manual identification of the road repair status in related technologies.

[0006] According to one aspect of an embodiment of the present application, a road identification method is provided, including: performing frame processing on a historical road damage video to obtain a plurality of road damage pictures, and determining a first picture of a road that has been repaired and is qualified, a second picture of a road that has been repaired and is unqualified, and a third picture of a road that has not been repaired from the plurality of road damage pictures; performing feature extraction on the first picture based on multiple dimensions of the road damage to obtain a plurality of first features, performing feature extraction on the second picture to obtain a plurality of second features, and performing feature extraction on the third picture to obtain a plurality of third features; combining the plurality of first features to obtain a first target feature of all dimensions of the road damage, combining the plurality of second features to obtain a second target feature of all dimensions of the road damage, and combining the plurality of third features to obtain a third target feature of all dimensions of the road damage; training a preset neural network model based on the first target feature, the second target feature, and the third target feature to obtain a trained target neural network model, and using the trained target neural network model to identify the road damage video to be identified collected from the target object to determine the road repair result.

[0007] Optionally, the multiple dimensions of road damage include at least two of the following: road cracks, road potholes, road sanding, road slope, road collapse, and road deformation.

[0008] Optionally, based on multiple dimensions of road damage, feature extraction is performed on the first image to obtain multiple first features, feature extraction is performed on the second image to obtain multiple second features, and feature extraction is performed on the third image to obtain multiple third features, including: dividing the first image into a first training set, a first validation set, and a first test set according to a first preset ratio; determining, based on the first training set, the first validation set, and the first test set, multiple first features corresponding to the first image based on multiple dimensions of road damage using a preset convolutional neural network model; dividing the second image into a second training set, a second validation set, and a second test set according to a second preset ratio; determining, based on the second training set, the second validation set, and the second test set, multiple second features corresponding to the second image based on multiple dimensions of road damage using a preset convolutional neural network model; dividing the third image into a third training set, a third validation set, and a third test set according to a third preset ratio; determining, based on the third training set, the third validation set, and the third test set, multiple third features corresponding to the third image based on multiple dimensions of road damage using a preset convolutional neural network model.

[0009] Optionally, before performing frame processing on the historical road damage video to obtain a plurality of road damage images, the method further includes: performing data cleaning on the historical road damage video.

[0010] Optionally, the plurality of first features, the plurality of second features and the plurality of third features each include at least: a length feature and a width feature.

[0011] Optionally, the road repair result includes: a first probability corresponding to the road being repaired and qualified, a second probability corresponding to the road being repaired and unqualified, and a third probability corresponding to the road being repaired and not being completed.

[0012] Optionally, when the second probability is greater than a preset threshold, the road damage video to be identified and the identification time corresponding to the road damage video to be identified are sent to the server side, and an alarm message is generated.

[0013] According to another aspect of the embodiments of the present application, a road identification device is also provided, including: a first determination module, used to frame-process historical road disease videos to obtain multiple road disease pictures, and determine a first picture of the road that has been repaired and is qualified, a second picture of the road that has been repaired but is unqualified, and a third picture of the road that has not been repaired from the multiple road disease pictures; an extraction module, used to perform feature extraction on the first picture based on multiple dimensions of the road disease to obtain multiple first features, perform feature extraction on the second picture to obtain multiple second features, and perform feature extraction on the third picture to obtain multiple third features; a combination module, used to combine the multiple first features to obtain a first target feature of all dimensions of the road disease, combine the multiple second features to obtain a second target feature of all dimensions of the road disease, and combine the multiple third features to obtain a third target feature of all dimensions of the road disease; a second determination module, used to train a preset neural network model based on the first target feature, the second target feature, and the third target feature to obtain a trained target neural network model and use the trained target neural network model to identify the road disease video to be identified collected from the target object, and determine the road repair result.

[0014] According to another aspect of the embodiments of the present application, a non-volatile storage medium is provided, which includes a stored program, wherein when the program is executed, the device where the storage medium is located is controlled to execute the above road identification method.

[0015] According to another aspect of the embodiments of the present application, a wearable electronic device is provided, including: a memory and a processor, wherein the processor is configured to run a program stored in the memory, wherein the above road recognition method is executed when the program is run.

[0016] In an embodiment of the present application, a historical road damage video is subjected to frame processing to obtain a plurality of road damage pictures, and a first picture of a road that has been repaired and is qualified, a second picture of a road that has been repaired and is unqualified, and a third picture of a road that has not been repaired are determined among the plurality of road damage pictures; based on the plurality of dimensions of the road damage, feature extraction is performed on the first picture to obtain a plurality of first features, feature extraction is performed on the second picture to obtain a plurality of second features, and feature extraction is performed on the third picture to obtain a plurality of third features; the plurality of first features are combined to obtain a first target feature of all dimensions of the road damage, the plurality of second features are combined to obtain a second target feature of all dimensions of the road damage, and the plurality of third features are combined to obtain a third target feature of all dimensions of the road damage; according to the first target feature, The preset neural network model is trained by using the historical road disease videos, the second target features and the third target features to obtain the trained target neural network model, and the trained target neural network model is used to identify the road disease videos to be identified collected by the target object, so as to determine the road repair result. By using the historical road disease videos to train the preset neural network model, and using the trained target neural network model to identify the road disease videos to be identified collected by the target object, the purpose of using the neural network model to identify the repair status of the road is achieved, thereby achieving the technical effect of improving the efficiency of road acceptance and quality inspection, and then solving the technical problem of low efficiency of road acceptance and quality inspection caused by the need for manual identification of the road repair status in related technologies. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0018] Figure 1 is a flow chart of a road identification method according to an embodiment of the present application;

[0019] Figure 2 is a structural diagram of a road identification device according to an embodiment of the present application;

[0020] Figure 3 This is a hardware structure block diagram of a computer terminal (or electronic device) according to a road identification method according to an embodiment of the present application. DETAILED DESCRIPTION

[0021] In order to enable those skilled in the art to better understand the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of this application.

[0022] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in a sequence other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0023] According to an embodiment of the present application, a method embodiment of a road identification method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0024] Figure 1 is a flow chart of a road identification method according to an embodiment of the present application, such as Figure 1 As shown, the method includes the following steps:

[0025] Step S102: The historical road damage video is frame-processed to obtain a plurality of road damage pictures, and a first picture of a road that has been repaired and is qualified, a second picture of a road that has been repaired but is unqualified, and a third picture of a road that has not been repaired are determined among the plurality of road damage pictures.

[0026] Road conditions include, but are not limited to, the following: pavement cracks, such as crazing, cracking, and cracks in the asphalt surface; potholes, caused by insufficient pavement bearing capacity, subsidence, or water intrusion; sanding and stoning, resulting in loose sand and gravel particles on the pavement surface; uneven road slope, leading to water accumulation and poor drainage; road collapse, resulting in subsidence or collapse of the roadbed or pavement; road deformation, such as wavy pavement, rutting, and wheel tracks; horizontal marking wear, resulting in worn, peeling, or blurred road markings; vertical marking wear, resulting in worn, damaged, or tilted road signs; drainage defects, such as clogged rainwater collection tanks and damaged drainage pipes; and green belt defects, such as dead vegetation and collapsed lawns. Road conditions not only affect driving safety and comfort but also increase vehicle wear and tear and repair costs, necessitating prompt detection and repair.

[0027] If the cracks and potholes on the road surface are completely repaired, the road can be deemed to have been repaired and the repair is qualified; if the cracks and potholes on the road surface are partially repaired but partly not repaired, the road can be deemed to have been repaired and the repair is unqualified; if the cracks and potholes on the road surface are not repaired at all, the road can be deemed to have not been repaired.

[0028] Optionally, the specific steps for performing frame processing on the historical road damage video are as follows:

[0029] 1. Import the required libraries and modules, such as OpenCV library, etc.

[0030] 2. Read the video file. You can use the `cv2.VideoCapture()` function in OpenCV to read the video.

[0031] Python

[0032] video=cv2.VideoCapture('video.mp4')

[0033] 3. Get the frame rate and total number of frames of the video.

[0034] Python

[0035] fps=video.get(cv2.CAP_PROP_FPS)

[0036] total_frames=int(video.get(cv2.CAP_PROP_FRAME_COUNT))

[0037] 4. Traverse each frame and decompose the video into frames.

[0038] Python

[0039] for frame_numin range(total_frames):

[0040] #Read the current frame

[0041] ret,frame=video.read()

[0042] #Perform frame processing operations here, such as image enhancement, target detection, etc.

[0043] #Save the processed frame

[0044] cv2.imwrite('frame_{}.jpg'.format(frame_num),frame)

[0045] 5. After processing all frames, release the video object.

[0046] Python

[0047] video.release()

[0048] Through the above steps, the video can be divided into frames, and then other operations such as image enhancement, target detection, feature extraction, etc. can be performed on each frame.

[0049] In step S104, based on multiple dimensions of road damage, feature extraction is performed on the first image to obtain multiple first features, feature extraction is performed on the second image to obtain multiple second features, and feature extraction is performed on the third image to obtain multiple third features.

[0050] For example, based on dimensions such as road cracks and potholes, feature extraction is performed on the first image to obtain the length and width features of the road cracks in the first image, as well as the length and width features of the road potholes; feature extraction is performed on the second image to obtain the length and width features of the road cracks in the second image, as well as the length and width features of the road potholes; feature extraction is performed on the third image to obtain the length and width features of the road cracks in the third image, as well as the length and width features of the road potholes.

[0051] Step S106: Combine multiple first features to obtain first target features of all dimensions of road damage, combine multiple second features to obtain second target features of all dimensions of road damage, and combine multiple third features to obtain third target features of all dimensions of road damage.

[0052] Optionally, the length features and width features of the pavement cracks in the first picture, as well as the length features and width features of the pavement potholes, are combined to obtain the first target features of the pavement cracks and pavement pothole dimensions corresponding to the first picture; the length features and width features of the pavement cracks in the second picture, as well as the length features and width features of the pavement potholes, are combined to obtain the second target features of the pavement cracks and pavement pothole dimensions corresponding to the second picture; the length features and width features of the pavement cracks in the third picture, as well as the length features and width features of the pavement potholes, are combined to obtain the third target features of the pavement cracks and pavement pothole dimensions corresponding to the third picture.

[0053] In step S108, the preset neural network model is trained according to the first target feature, the second target feature, and the third target feature to obtain a trained target neural network model. The trained target neural network model is used to identify the road disease video to be identified collected by the target object to determine the road repair result.

[0054] Pre-defined neural network models include, but are not limited to, capsule networks. Capsule networks, also known as capsule neural networks, are a deep learning model designed to address some of the limitations of traditional convolutional neural networks (CNNs) in processing spatial relationships. Traditional CNN models use convolutional and pooling layers to extract features and fully connected layers for classification. However, CNNs are not robust to variations in object pose, rotation, and deformation when processing spatial relationships. Furthermore, CNNs lack a robust method for capturing hierarchical relationships between instances when processing multiple instances. Capsule networks attempt to address these issues by introducing the concept of capsules. A capsule is a set of vectors that contain the pose information and presence probability of an entity. Capsule networks use a dynamic routing algorithm to calculate weights between capsules to capture hierarchical relationships between entities. This enables capsule networks to better handle variations in object deformation, rotation, and pose. In capsule networks, the output of a capsule represents the presence probability of a feature and its pose information. Each capsule is connected to all capsules in the previous layer, and the weights between them are calculated using a dynamic routing algorithm. The dynamic routing algorithm updates the weights based on the consistency between the two capsules, that is, whether the pose information output by the capsule is consistent with the prediction of the previous layer input. This enables the capsule network to capture the hierarchical relationship between different instances.

[0055] According to the above steps, by using historical road disease videos to train the preset neural network model, and using the trained target neural network model to identify the road disease videos to be identified collected by the target object, the purpose of using the neural network model to identify the repair status of the road is achieved, thereby achieving the technical effect of improving the efficiency of road acceptance and quality inspection.

[0056] According to some optional embodiments of the present application, the multiple dimensions of road damage include at least two of the following: road cracks, road potholes, road sanding, road slope, road collapse, and road deformation.

[0057] According to some other optional embodiments of the present application, based on multiple dimensions of road damage, feature extraction is performed on the first image to obtain multiple first features, feature extraction is performed on the second image to obtain multiple second features, and feature extraction is performed on the third image to obtain multiple third features, including the following steps: dividing the first image into a first training set, a first validation set, and a first test set according to a first preset ratio; determining the multiple first features corresponding to the first image based on the multiple dimensions of road damage according to the first training set, the first validation set, and the first test set using a preset convolutional neural network model; dividing the second image into a second training set, a second validation set, and a second test set according to a second preset ratio; determining the multiple second features corresponding to the second image based on the multiple dimensions of road damage according to the second training set, the second validation set, and the second test set using a preset convolutional neural network model; dividing the third image into a third training set, a third validation set, and a third test set according to a third preset ratio; determining the multiple third features corresponding to the third image based on the multiple dimensions of road damage according to the third training set, the third validation set, and the third test set using a preset convolutional neural network model.

[0058] It can be understood that the roles of the training set, validation set, and test set in image feature extraction are as follows: the training set is used to train the feature extraction model. By extracting image features in the training set, the model can learn useful features in the image. The training set usually contains a large number of samples to ensure that the model can learn features of different categories and has sufficient generalization ability. The validation set is used to adjust the hyperparameters and model structure of the feature extraction model. By extracting image features in the validation set and evaluating the model performance, the optimal hyperparameter settings and model structure can be selected. The validation set is usually independent of the training set to ensure that the evaluation of the model is objective and accurate. The test set is used to evaluate the performance of the feature extraction model. By extracting image features in the test set and evaluating the prediction accuracy of the model, the performance of the model in actual applications can be obtained. Optionally, the above preset ratio is, for example: 6:2:2.

[0059] The following are detailed steps for extracting features from the first, second, and third images using a preset convolutional neural network model: First, preprocess the images in the training, test, and validation sets. This preprocessing includes, but is not limited to, image normalization, cropping, and resizing to ensure consistent size and format of the input images. Second, use the preset convolutional neural network model to extract features from the images. The output of the model's convolutional layer can be used as the feature representation of the image. The images in the training set are input into the model to extract and save the features. The extracted features are then converted to vector representations. For example, dimensionality reduction or linear discriminant analysis can be used to reduce high-dimensional features to low dimensions to reduce computational complexity. The feature vectors and corresponding labels of the training set are used to train a classification model. Finally, the feature vectors and corresponding labels of the test set are used to evaluate the model's performance. For example, metrics such as accuracy, precision, and recall can be calculated to assess the model's classification performance. Finally, based on the feature vectors and corresponding labels of the validation set, model hyperparameters, such as the regularization parameter and learning rate, are tuned. Methods such as cross-validation can also be used to select the optimal hyperparameter combination.

[0060] In some optional embodiments of the present application, before the historical road damage video is frame-processed to obtain a plurality of road damage images, the method further includes: performing data cleaning on the historical road damage video.

[0061] Optionally, the specific steps for data cleaning historical road damage videos are as follows: 1. Data Filtering: Filter the collected data to remove noise, errors, or unnecessary information. This can be achieved through the use of techniques such as filters and thresholds. 2. Data Repair: Repair missing or damaged data in the video. This can include using interpolation algorithms to fill in missing frames or repair damaged pixels. 3. Data Proofreading: Proofread the cleaned data to ensure its accuracy and consistency. This can be achieved by comparing it with other data sources.

[0062] As some optional embodiments of the present application, the plurality of first features, the plurality of second features, and the plurality of third features all include at least: a length feature and a width feature.

[0063] In some optional embodiments of the present application, the road repair results include: a first probability corresponding to the road being repaired and qualified, a second probability corresponding to the road being repaired and unqualified, and a third probability corresponding to the road being repaired and not being qualified.

[0064] As other optional embodiments of the present application, when the second probability is greater than a preset threshold, the road damage video to be identified and the identification time corresponding to the road damage video to be identified are sent to the server side, and an alarm message is generated.

[0065] The above steps can help inspection personnel to intelligently judge the road maintenance status during the inspection process, greatly improving the efficiency of road maintenance work.

[0066] Figure 2 is a structural diagram of a road identification device according to an embodiment of the present application, such as Figure 2 As shown, the device includes:

[0067] The first determination module 20 is used to frame-process the historical road damage video to obtain multiple road damage pictures, and determine a first picture of the road that has been repaired and is repaired to a qualified standard, a second picture of the road that has been repaired but is repaired to an unqualified standard, and a third picture of the road that has not been repaired.

[0068] Road conditions include, but are not limited to, the following: pavement cracks, such as crazing, cracking, and cracks in the asphalt surface; potholes, caused by insufficient pavement bearing capacity, subsidence, or water intrusion; sanding and stoning, resulting in loose sand and gravel particles on the pavement surface; uneven road slope, leading to water accumulation and poor drainage; road collapse, resulting in subsidence or collapse of the roadbed or pavement; road deformation, such as wavy pavement, rutting, and wheel tracks; horizontal marking wear, resulting in worn, peeling, or blurred road markings; vertical marking wear, resulting in worn, damaged, or tilted road signs; drainage defects, such as clogged rainwater collection tanks and damaged drainage pipes; and green belt defects, such as dead vegetation and collapsed lawns. Road conditions not only affect driving safety and comfort but also increase vehicle wear and tear and repair costs, necessitating prompt detection and repair.

[0069] If the cracks and potholes on the road surface are completely repaired, the road can be deemed to have been repaired and the repair is qualified; if the cracks and potholes on the road surface are partially repaired but partly not repaired, the road can be deemed to have been repaired and the repair is unqualified; if the cracks and potholes on the road surface are not repaired at all, the road can be deemed to have not been repaired.

[0070] Optionally, the specific steps for performing frame processing on the historical road damage video are as follows:

[0071] 1. Import the required libraries and modules, such as OpenCV library, etc.

[0072] 2. Read the video file. You can use the `cv2.VideoCapture()` function in OpenCV to read the video.

[0073] Python

[0074] video=cv2.VideoCapture('video.mp4')

[0075] 3. Get the frame rate and total number of frames of the video.

[0076] Python

[0077] fps=video.get(cv2.CAP_PROP_FPS)

[0078] total_frames=int(video.get(cv2.CAP_PROP_FRAME_COUNT))

[0079] 4. Traverse each frame and decompose the video into frames.

[0080] Python

[0081] for frame_numin range(total_frames):

[0082] #Read the current frame

[0083] ret,frame=video.read()

[0084] #Perform frame processing operations here, such as image enhancement, target detection, etc.

[0085] #Save the processed frame

[0086] cv2.imwrite('frame_{}.jpg'.format(frame_num),frame)

[0087] 5. After processing all frames, release the video object.

[0088] Python

[0089] video.release()

[0090] The extraction module 22 is used to perform feature extraction on the first image based on multiple dimensions of road damage to obtain multiple first features, perform feature extraction on the second image to obtain multiple second features, and perform feature extraction on the third image to obtain multiple third features.

[0091] For example, based on dimensions such as road cracks and potholes, feature extraction is performed on the first image to obtain the length and width features of the road cracks in the first image, as well as the length and width features of the road potholes; feature extraction is performed on the second image to obtain the length and width features of the road cracks in the second image, as well as the length and width features of the road potholes; feature extraction is performed on the third image to obtain the length and width features of the road cracks in the third image, as well as the length and width features of the road potholes.

[0092] The combination module 24 is used to combine multiple first features to obtain the first target features of all dimensions of road diseases, combine multiple second features to obtain the second target features of all dimensions of road diseases, and combine multiple third features to obtain the third target features of all dimensions of road diseases.

[0093] Optionally, the length features and width features of the pavement cracks in the first picture, as well as the length features and width features of the pavement potholes, are combined to obtain the first target features of the pavement cracks and pavement pothole dimensions corresponding to the first picture; the length features and width features of the pavement cracks in the second picture, as well as the length features and width features of the pavement potholes, are combined to obtain the second target features of the pavement cracks and pavement pothole dimensions corresponding to the second picture; the length features and width features of the pavement cracks in the third picture, as well as the length features and width features of the pavement potholes, are combined to obtain the third target features of the pavement cracks and pavement pothole dimensions corresponding to the third picture.

[0094] The second determination module 26 is used to train the preset neural network model according to the first target feature, the second target feature and the third target feature to obtain a trained target neural network model and use the trained target neural network model to identify the road disease video to be identified collected by the target object, and determine the road repair result.

[0095] Pre-defined neural network models include, but are not limited to, capsule networks. Capsule networks, also known as capsule neural networks, are a deep learning model designed to address some of the limitations of traditional convolutional neural networks (CNNs) in processing spatial relationships. Traditional CNN models use convolutional and pooling layers to extract features and fully connected layers for classification. However, CNNs are not robust to variations in object pose, rotation, and deformation when processing spatial relationships. Furthermore, CNNs lack a robust method for capturing hierarchical relationships between instances when processing multiple instances. Capsule networks attempt to address these issues by introducing the concept of capsules. A capsule is a set of vectors that contain the pose information and presence probability of an entity. Capsule networks use a dynamic routing algorithm to calculate weights between capsules to capture hierarchical relationships between entities. This enables capsule networks to better handle variations in object deformation, rotation, and pose. In capsule networks, the output of a capsule represents the presence probability of a feature and its pose information. Each capsule is connected to all capsules in the previous layer, and the weights between them are calculated using a dynamic routing algorithm. The dynamic routing algorithm updates the weights based on the consistency between the two capsules, that is, whether the pose information output by the capsule is consistent with the prediction of the previous layer input. This enables the capsule network to capture the hierarchical relationship between different instances.

[0096] It should be noted that the above Figure 2 The modules in the embodiment can be program modules (for example, a set of program instructions that implement a specific function) or hardware modules. For the latter, they can be expressed in the following forms, but are not limited to these: the expression form of each of the above modules is a processor, or the functions of each of the above modules are implemented by a processor.

[0097] It should be noted that Figure 2 The preferred implementation of the embodiment shown can be found in Figure 1 The relevant description of the illustrated embodiment will not be repeated here.

[0098] Figure 3 FIG1 shows a hardware structure block diagram of a computer terminal (or mobile device) for implementing a road recognition method. Figure 3As shown, the computer terminal 30 (or mobile device) may include one or more (illustrated as 302a, 302b, ..., 302n in the figure) processors 302 (the processor 302 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 304 for storing data, and a transmission module 306 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the BUS bus), a network interface, a power supply and / or a camera. It will be understood by those skilled in the art that Figure 3 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 3 More or fewer components than shown, or with Figure 3 Different configurations shown.

[0099] It should be noted that the one or more processors 302 and / or other data processing circuits described above may generally be referred to herein as "data processing circuitry". The data processing circuitry may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. In addition, the data processing circuitry may be a single independent processing module, or may be incorporated in whole or in part into any of the other components in the computer terminal 30 (or mobile device). As described in the embodiments of the present application, the data processing circuitry serves as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0100] Memory 304 can be used to store software programs and modules for application software, such as the program instructions / data storage device corresponding to the road identification method in the embodiments of the present application. Processor 302 executes the software programs and modules stored in memory 304 to perform various functional applications and data processing, thereby implementing the aforementioned road identification method. Memory 304 can include high-speed random access memory (RAM) and non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, memory 304 may further include memory remotely located from processor 302, which can be connected to computer terminal 30 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0101] The transmission module 306 is configured to receive or transmit data via a network. A specific example of the aforementioned network may include a wireless network provided by the communications provider of the computer terminal 30. In one embodiment, the transmission module 306 includes a network interface controller (NIC), which can be connected to other network devices via a base station to enable communication with the Internet. In another embodiment, the transmission module 306 may be a radio frequency (RF) module, which is configured to communicate with the Internet wirelessly.

[0102] The display may be, for example, a touch screen liquid crystal display (LCD) that enables a user to interact with a user interface of the computer terminal 30 (or mobile device).

[0103] It should be noted that, in some optional embodiments, the above Figure 3 The computer device (or electronic device) shown may include hardware elements (including circuits), software elements (including computer code stored on a computer-readable medium), or a combination of hardware elements and software elements. Figure 3 This is merely one example of a particular embodiment and is intended to illustrate the types of components that may be present in the computer device (or electronic device) described above.

[0104] It should be noted that Figure 3 The electronic device shown is used to perform Figure 1 The road identification method shown in the figure, therefore the relevant explanations in the execution method of the above command also apply to the electronic device and will not be repeated here.

[0105] An embodiment of the present application further provides a non-volatile storage medium, which includes a stored program, wherein when the program is running, the device where the storage medium is located is controlled to execute the above road identification method.

[0106] The non-volatile storage medium performs a program for performing the following functions: performing frame processing on a historical road damage video to obtain a plurality of road damage images, and determining a first image of a road that has been repaired and is qualified, a second image of a road that has been repaired and is unqualified, and a third image of a road that has not been repaired from the plurality of road damage images; performing feature extraction on the first image based on multiple dimensions of the road damage to obtain a plurality of first features, performing feature extraction on the second image to obtain a plurality of second features, and performing feature extraction on the third image to obtain a plurality of third features; combining the plurality of first features to obtain a first target feature of all dimensions of the road damage, combining the plurality of second features to obtain a second target feature of all dimensions of the road damage, and combining the plurality of third features to obtain a third target feature of all dimensions of the road damage; training a preset neural network model based on the first target feature, the second target feature, and the third target feature to obtain a trained target neural network model, and using the trained target neural network model to identify the road damage video to be identified collected from the target object to determine the road repair result.

[0107] An embodiment of the present application further provides a wearable electronic device, comprising: a memory and a processor, wherein the processor is configured to run a program stored in the memory, wherein the above road recognition method is executed when the program is run.

[0108] The processor is used to run a program that performs the following functions: frame-processing historical road damage videos to obtain multiple road damage pictures, and determining a first picture of a road that has been repaired and is qualified, a second picture of a road that has been repaired and is unqualified, and a third picture of a road that has not been repaired among the multiple road damage pictures; based on multiple dimensions of road damage, feature extraction is performed on the first picture to obtain multiple first features, feature extraction is performed on the second picture to obtain multiple second features, and feature extraction is performed on the third picture to obtain multiple third features; the multiple first features are combined to obtain a first target feature of all dimensions of road damage, the multiple second features are combined to obtain a second target feature of all dimensions of road damage, and the multiple third features are combined to obtain a third target feature of all dimensions of road damage; based on the first target feature, the second target feature, and the third target feature, a preset neural network model is trained to obtain a trained target neural network model, and the trained target neural network model is used to identify the road damage video to be identified collected from the target object to determine the road repair result.

[0109] Using these wearable electronic devices, road maintenance personnel can conduct simultaneous inspections and acceptance checks, shortening the maintenance cycle and improving the efficiency of repair feedback. This provides data support for the formulation of maintenance funding and annual maintenance plans. Furthermore, these wearable electronic devices transmit identification records to cloud servers, helping management units to document and record acceptance work, achieving digital maintenance.

[0110] The serial numbers of the above embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0111] In the above embodiments of the present application, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0112] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only exemplary. For example, the division of the units can be a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0113] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0114] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0115] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the relevant technology or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, a server or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0116] The above is only a preferred embodiment of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A road identification method, characterized in that: include: Processing the historical road damage video by frame division to obtain a plurality of road damage images, and determining, from the plurality of road damage images, a first image of a road that has been repaired and has passed the repair, a second image of a road that has been repaired and has failed the repair, and a third image of a road that has not been repaired; Based on multiple dimensions of road damage, feature extraction is performed on the first image to obtain multiple first features, feature extraction is performed on the second image to obtain multiple second features, and feature extraction is performed on the third image to obtain multiple third features; Combining the multiple first features to obtain a first target feature of all dimensions of road damage, combining the multiple second features to obtain a second target feature of all dimensions of road damage, and combining the multiple third features to obtain a third target feature of all dimensions of road damage; According to the first target feature, the second target feature and the third target feature, the preset neural network model is trained to obtain a trained target neural network model, and the trained target neural network model is used to identify the road disease video to be identified collected from the target object to determine the road repair result.

2. The method according to claim 1, characterized in that The multiple dimensions of road damage include at least two of the following: pavement cracks, pavement potholes, pavement sanding, pavement slope, pavement collapse, and pavement deformation.

3. The method according to claim 2, characterized in that Based on multiple dimensions of road damage, feature extraction is performed on the first image to obtain multiple first features, feature extraction is performed on the second image to obtain multiple second features, and feature extraction is performed on the third image to obtain multiple third features, including: Dividing the first images into a first training set, a first validation set, and a first test set according to a first preset ratio; Determining, based on the first training set, the first validation set, and the first test set, the plurality of first features corresponding to the first image using a preset convolutional neural network model and based on multiple dimensions of road damage; Dividing the second images into a second training set, a second validation set, and a second test set according to a second preset ratio; Determining, based on the second training set, the second validation set, and the second test set, the plurality of second features corresponding to the second image using a preset convolutional neural network model and based on multiple dimensions of road damage; Dividing the third images into a third training set, a third validation set, and a third test set according to a third preset ratio; According to the third training set, the third validation set and the third test set, using a preset convolutional neural network model, based on multiple dimensions of road damage, the multiple third features corresponding to the third image are determined.

4. The method according to claim 1, wherein Before performing frame processing on the historical road damage video to obtain a plurality of road damage pictures, the method further includes: performing data cleaning on the historical road damage video.

5. The method according to claim 1, wherein The plurality of first features, the plurality of second features, and the plurality of third features each include at least a length feature and a width feature.

6. The method according to claim 1, wherein The road repair result includes: a first probability corresponding to the road being repaired and qualified, a second probability corresponding to the road being repaired and unqualified, and a third probability corresponding to the road being repaired and not being completed.

7. The method according to claim 6, characterized in that When the second probability is greater than a preset threshold, the road damage video to be identified and the identification time corresponding to the road damage video to be identified are sent to the server side, and an alarm message is generated.

8. A road recognition device, characterized in that: include: A first determination module is configured to perform frame processing on a historical road damage video to obtain a plurality of road damage images, and to determine, from the plurality of road damage images, a first image of a road that has been repaired and has passed the repair, a second image of a road that has been repaired and has failed the repair, and a third image of a road that has not been repaired; an extraction module, configured to perform feature extraction on the first image based on multiple dimensions of road damage to obtain multiple first features, perform feature extraction on the second image to obtain multiple second features, and perform feature extraction on the third image to obtain multiple third features; a combining module, configured to combine the plurality of first features to obtain a first target feature of all dimensions of road damage, combine the plurality of second features to obtain a second target feature of all dimensions of road damage, and combine the plurality of third features to obtain a third target feature of all dimensions of road damage; The second determination module is used to train a preset neural network model based on the first target feature, the second target feature and the third target feature to obtain a trained target neural network model and use the trained target neural network model to identify the road disease video to be identified collected from the target object, and determine the road repair result.

9. A non-volatile storage medium, characterized in that: The non-volatile storage medium includes a stored program, wherein when the program is executed, the device where the non-volatile storage medium is located is controlled to execute the road identification method according to any one of claims 1 to 7.

10. A wearable electronic device, characterized in that: include: A memory and a processor, wherein the processor is configured to run a program stored in the memory, wherein the program, when run, executes the road recognition method according to any one of claims 1 to 7.

Images