A road asset detection model construction and road asset detection method

Abstract

The present invention discloses a road asset detection model construction and road asset detection method, which includes obtaining a road asset image dataset; classifying images in the road asset image dataset according to target attributes and inputting the classified labeled image sample set into a preset neural network for training until a road asset detection model that meets the requirements is obtained; extracting frames from the acquired video stream data to obtain a corresponding road asset image dataset and inputting the dataset into the road asset detection model for training. This method classifies images in the image dataset according to target attributes, which helps the model infer attributes of different labels, while facilitating model management and deployment, and enabling comprehensive and accurate understanding of road asset images. The trained road asset detection model is then used to detect the input road asset image data, replacing manual detection and improving detection efficiency and accuracy.

Description

Technical Field

[0001] The present invention relates to the technical field of road asset management, and in particular to a road asset detection model construction and a road asset detection method. Background Art

[0002] The management of road assets (highways, railways, etc.) involves both inspection and maintenance. Traditional inspection methods rely on manual patrols, which are not only time-consuming and labor-intensive but also prone to omissions. Management units are unable to fully and accurately understand the status of their road assets, nor are they able to fully and accurately track damage or loss. With the rapid development of highways and railways, this traditional road asset inspection method is increasingly showing its limitations. Summary of the Invention

[0003] In view of this, embodiments of the present invention provide a road asset detection model construction and a road asset detection method to solve the technical problem that traditional road asset detection methods in the prior art have high limitations.

[0004] The technical solutions proposed by the present invention are as follows:

[0005] A first aspect of an embodiment of the present invention provides a method for constructing a road asset detection model, which includes: obtaining a road asset image dataset; classifying images in the road asset image dataset according to target attributes and inputting the classified labeled image sample set into a preset neural network for training until a road asset detection model that meets the conditions is obtained.

[0006] Optionally, after obtaining the road asset image dataset, the method further includes: performing data enhancement processing on the road asset image dataset to obtain an enhanced road asset image dataset; and using images in the enhanced road asset image dataset for training to obtain the road asset detection model.

[0007] Optionally, the road asset image dataset includes a lane line dataset and a road marking dataset.

[0008] A second aspect of an embodiment of the present invention provides a road asset detection method, which includes: obtaining video stream data; performing frame extraction processing on the video stream data to obtain corresponding road asset image data; and inputting the road asset image data into a road asset detection model constructed by the road asset detection model construction method described in the first aspect of the embodiment of the present invention and any one of the first aspects for detection.

[0009] A third aspect of an embodiment of the present invention provides a road asset detection model construction device, which includes: a first acquisition module for acquiring a road asset image dataset; a training module for classifying images in the road asset image dataset according to target attributes and inputting the classified labeled image sample set into a preset neural network for training until a road asset detection model that meets the conditions is obtained.

[0010] Optionally, the device also includes: a first processing module, used to perform data enhancement processing on the road asset image dataset to obtain an enhanced road asset image dataset; and a first training module, used to train the road asset detection model using images in the enhanced road asset image dataset.

[0011] Optionally, the road asset image dataset includes a lane line dataset and a road marking dataset.

[0012] A fourth aspect of an embodiment of the present invention provides a road asset detection device, which includes: a second acquisition module for acquiring video stream data; a second processing module for performing frame extraction processing on the video stream data to obtain corresponding road asset image data; and a detection module for inputting the road asset image data into a road asset detection model constructed by the road asset detection model construction method described in the first aspect and any one of the first aspects of the embodiment of the present invention for detection.

[0013] A fifth aspect of an embodiment of the present invention provides a computer-readable storage medium, which stores computer instructions, and the computer instructions are used to enable the computer to execute the road asset detection model construction method as described in the first aspect of the embodiment of the present invention and any one of the first aspects, or the road asset detection method as described in the second aspect of the embodiment of the present invention.

[0014] A sixth aspect of an embodiment of the present invention provides an electronic device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the computer instructions to execute the road asset detection model construction method as described in the first aspect of the embodiment of the present invention and any one of the first aspects, or the road asset detection method as described in the second aspect of the embodiment of the present invention.

[0015] The technical solution provided by the present invention has the following effects:

[0016] An embodiment of the present invention provides a method for constructing a road asset detection model. The method comprises: obtaining a road asset image dataset; classifying images in the dataset according to target attributes; and inputting the classified, labeled image samples into a preset neural network for training until a qualified road asset detection model is obtained. This method classifies images in the dataset according to target attributes, facilitating the model's inference of attributes for different labels and facilitating model management and deployment. Therefore, the present invention enables a comprehensive and accurate understanding of road asset images.

[0017] An embodiment of the present invention provides a road asset detection method, comprising: acquiring video stream data; extracting frames from the video stream data to obtain corresponding road asset image data; and inputting the road asset image data into a road asset detection model constructed using the road asset detection model construction method described in an embodiment of the present invention for detection. This method utilizes a trained road asset detection model to detect the input road asset image data, replacing manual detection and improving detection efficiency and accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the specific embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0019] Figure 1 is a flowchart of a method for constructing a road asset detection model according to an embodiment of the present invention;

[0020] Figure 2 is a flow chart of a road asset detection method according to an embodiment of the present invention;

[0021] Figure 3 is a structural block diagram of a device for constructing a road asset detection model according to an embodiment of the present invention;

[0022] Figure 4 is a structural block diagram of a road asset detection device according to an embodiment of the present invention;

[0023] Figure 5 is a schematic diagram of the structure of a computer-readable storage medium provided according to an embodiment of the present invention;

[0024] Figure 6 is a schematic structural diagram of an electronic device provided according to an embodiment of the present invention. DETAILED DESCRIPTION

[0025] To make the purpose, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative efforts shall fall within the scope of protection of the present invention.

[0026] The embodiment of the present invention provides a method for constructing a road asset detection model. Figure 1 As shown, the method includes the following steps:

[0027] Step S101: Acquire a road asset image dataset. Specifically, image sensors such as vehicle-mounted cameras or fixed cameras at intersections can be used to capture images of road assets in actual traffic scenarios to form a road asset image dataset. The road asset image dataset includes multi-angle samples (multi-angle road asset images) collected from multiple scenes and time periods. The collected road asset images must be clear and easily distinguishable, for example, with pixel values ​​greater than 100*100. Road assets represent infrastructure assets formed through construction, such as roads and railways.

[0028] Step S102: Classify the images in the road asset image dataset according to target attributes and input the classified labeled image sample set into a preset neural network for training until a road asset detection model that meets the conditions is obtained. Specifically, the road asset image dataset contains images corresponding to multiple road assets. First, the road asset images in the road asset image dataset are classified according to the attributes of the road assets. For example, when the road asset image dataset includes road sign images and lane line images, since road signs may include left turn, straight ahead, right turn, left turn + straight ahead, right turn + straight ahead, left turn + U-turn, U-turn, left turn + right turn, zebra crossing, bus-only area, guide line, grid line, speed reduction, avoidance, stop line, signal light, guardrail, etc., lane lines may include white solid line, white dashed line, yellow solid line, yellow dashed line, double white solid line, double white dashed line, double yellow solid line, double yellow dashed line, white left dashed right solid line, white left solid right dashed line, yellow left dashed right solid line, yellow left solid right dashed line, minimum speed limit, maximum speed limit, etc. At this point, after classifying the images in the road asset image dataset according to the target attributes, we can obtain an image containing each road sign (road sign rectangular box) and an image containing each lane line (lane line point set), that is, labeled road sign images and labeled lane line images.

[0029] Finally, the resulting labeled image sample set is fed into a pre-defined neural network (e.g., a convolutional neural network like a CNN) for training until a road asset detection model that meets the requirements is obtained. The specific training process includes four steps: dataset preparation and cleaning, data labeling, network training, and testing.

[0030] For example, the LaneATTPro detection model is used to train the input lane line image until a lane line detection model that meets the conditions is obtained; the yoloX detection model is used to train the input road sign image until a road sign detection model that meets the conditions is obtained. Among them, the LaneATTPro detection model represents a lane line detection model that achieves detection by combining local and global features, making it easier to use information from other lanes when there are occlusions or no visible lane markings. Therefore, this method can improve the generalization of the lane line detection model and road sign detection model in actual application scenarios.

[0031] An embodiment of the present invention provides a method for constructing a road asset detection model. The method comprises: obtaining a road asset image dataset; classifying images in the dataset according to target attributes; and inputting the classified, labeled image samples into a preset neural network for training until a qualified road asset detection model is obtained. This method classifies images in the dataset according to target attributes, facilitating the model's inference of attributes for different labels and facilitating model management and deployment. Therefore, the present invention enables a comprehensive and accurate understanding of road asset images.

[0032] As an optional implementation of an embodiment of the present invention, after obtaining the road asset image dataset, the method further includes: performing data enhancement processing on the road asset image dataset to obtain an enhanced road asset image dataset; and using the images in the enhanced road asset image dataset for training to obtain the road asset detection model. Specifically, after obtaining the road asset image dataset, data enhancement processing is performed on the road asset image dataset to expand the dataset, so that more equally effective data can be obtained. The expanded dataset is then used for training, and the obtained road asset detection model has better effects and better generalization capabilities. Among them, data enhancement methods may include mosaic enhancement, random erasure, noise addition, etc., which are not specifically limited in the present invention, and it is sufficient to meet the data enhancement requirements.

[0033] As an optional implementation of this embodiment of the present invention, the road asset image dataset includes a lane line dataset and a road sign dataset. Specifically, the lane line images and road sign images contained in the lane line dataset and the road sign dataset are the same as the images provided in step S102 and are not further described here.

[0034] The embodiment of the present invention also provides a road asset detection method, such as Figure 2 As shown, the method includes the following steps:

[0035] Step S201: Acquire video stream data. Specifically, similar to step S101, sensors such as vehicle-mounted cameras or fixed cameras at intersections can be used to collect video streams in actual traffic scenes.

[0036] Step S202: Frame extraction is performed on the video stream data to obtain corresponding road asset image data. Specifically, the corresponding road asset image data can be obtained by extracting the obtained video stream data using methods such as OpenCV or FFmpeg. For example, if the video stream data contains multiple images such as road asset images and pedestrian images, the road asset images can be obtained by frame extraction and the corresponding road asset image data can be generated.

[0037] Step S203: Input the road asset image data into the road asset detection model constructed by the road asset detection model construction method described in the embodiment of the present invention for detection. Specifically, the road asset image data is input into the trained model for detection. For example, inputting lane line image data into the lane line detection model can obtain a lane line point set, through which the corresponding lane line can be determined; inputting road sign image data into the road sign detection model can obtain the coordinates of the corresponding road sign (the upper left and lower two points of the road sign rectangular box), category information, and confidence level, etc., and the corresponding road sign can be determined through this information.

[0038] Secondly, by inputting road asset image data into a trained road asset detection model for detection, it can also be used to detect motor vehicle and non-motor vehicle violations. For example, by detecting lane lines, the corresponding vehicle's route can be determined, and by detecting road signs, the road signs passed by the corresponding vehicle during the corresponding route can be determined. For example, through precise model detection, it is determined that Route A includes two road signs: left turn and straight ahead, and these signs are stored. When a vehicle is driving on Route A and is detected to have made a right turn, it can be determined that the vehicle has violated the driving regulations by comparing it with the stored road signs.

[0039] An embodiment of the present invention provides a road asset detection method, comprising: acquiring video stream data; extracting frames from the video stream data to obtain corresponding road asset image data; and inputting the road asset image data into a road asset detection model constructed using the road asset detection model construction method described in an embodiment of the present invention for detection. This method utilizes a trained road asset detection model to detect the input road asset image data, replacing manual detection and improving detection efficiency and accuracy.

[0040] The embodiment of the present invention also provides a road asset detection model construction device, such as Figure 3 As shown, the device includes:

[0041] The first acquisition module 301 is used to acquire a road asset image dataset; for details, please refer to the relevant description of step S101 in the above method embodiment.

[0042] The training module 302 is used to classify the images in the road asset image dataset according to target attributes and input the classified labeled image sample set into a preset neural network for training until a road asset detection model that meets the conditions is obtained; for details, please refer to the relevant description of step S102 in the above method embodiment.

[0043] The road asset detection model construction device provided by the embodiments of the present invention classifies images in an image dataset according to target attributes, helping the model infer the attributes of different labels while also facilitating model management and deployment. Therefore, the present invention enables a comprehensive and accurate understanding of road asset images.

[0044] As an optional implementation of an embodiment of the present invention, the device also includes: a first processing module, used to perform data enhancement processing on the road asset image dataset to obtain an enhanced road asset image dataset; and a first training module, used to train the road asset detection model using images in the enhanced road asset image dataset.

[0045] As an optional implementation of the embodiment of the present invention, the road asset image dataset includes a lane line dataset and a road sign dataset.

[0046] For a detailed description of the functions of the road asset detection model construction device provided in the embodiment of the present invention, please refer to the description of the road asset detection model construction method in the above embodiment.

[0047] The embodiment of the present invention also provides a road asset detection device, such as Figure 4 As shown, the device includes:

[0048] The second acquisition module 401 is used to acquire video stream data; for details, please refer to the relevant description of step S201 in the above method embodiment.

[0049] The second processing module 402 is used to perform frame extraction processing on the video stream data to obtain corresponding road asset image data; for details, please refer to the relevant description of step S202 in the above method embodiment.

[0050] The detection module 403 is used to input the road asset image data into the road asset detection model constructed by the road asset detection model construction method described in the embodiment of the present invention for detection; for details, please refer to the relevant description of step S203 in the above method embodiment.

[0051] The road asset detection device provided by the embodiment of the present invention uses a trained road asset detection model to detect input road asset image data, replacing manual detection and improving detection efficiency and accuracy.

[0052] For a detailed description of the functions of the road asset detection device provided in the embodiment of the present invention, please refer to the description of the road asset detection method in the above embodiment.

[0053] The embodiment of the present invention also provides a storage medium, such as Figure 5 As shown, a computer program 501 is stored thereon. When executed by a processor, the instructions implement the steps of the road asset detection model construction method or road asset detection method in the above-mentioned embodiment. The storage medium also stores audio and video stream data, feature frame data, interaction request signaling, encrypted data, and preset data size. The storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), a random access memory (RAM), a flash memory, a hard disk drive (HDD), or a solid-state drive (SSD); the storage medium can also include a combination of the above-mentioned types of memory.

[0054] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), a random access memory (RAM), a flash memory, a hard disk drive (HDD), or a solid-state drive (SSD). The storage medium can also include a combination of the above-mentioned types of memory.

[0055] The embodiment of the present invention further provides an electronic device, such as Figure 6 As shown, the electronic device may include a processor 61 and a memory 62, wherein the processor 61 and the memory 62 may be connected via a bus or other means. Figure 6 The bus connection is taken as an example.

[0056] The processor 61 may be a central processing unit (CPU). The processor 61 may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or a combination of the above chips.

[0057] Memory 62, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer executable programs, and modules, such as the corresponding program instructions / modules in the embodiments of the present invention. Processor 61 executes the non-transitory software programs, instructions, and modules stored in memory 62 to perform various processor functions and data processing, thereby implementing the road asset detection model construction method or road asset detection method in the above-mentioned method embodiments.

[0058] The memory 62 may include a program storage area and a data storage area, wherein the program storage area may store applications required for operating the device and at least one function; the data storage area may store data created by the processor 61, etc. In addition, the memory 62 may include a high-speed random access memory and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 62 may optionally include a memory remotely located relative to the processor 61, and these remote memories may be connected to the processor 61 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0059] The one or more modules are stored in the memory 62 and when executed by the processor 61, perform the following steps: Figure 1 -2 is a road asset detection model construction method or a road asset detection method in the embodiment shown in FIG.

[0060] For details of the above electronic equipment, please refer to Figures 1 to 2 The corresponding descriptions and effects in the embodiments shown can be understood and will not be repeated here.

[0061] Although the embodiments of the present invention have been described with reference to the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention. Such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A road asset detection method, characterized in that: The steps include: Acquire a road asset image dataset, wherein the road asset image dataset includes multi-angle road sign images and multi-angle lane line images collected from multiple scenes and multiple time periods; Classifying images in the road asset image dataset according to target attributes and inputting the classified labeled image sample set into a preset neural network for training until a road asset detection model that meets the conditions is obtained, wherein the classified labeled image sample set includes labeled road sign images and labeled lane line images, and the road asset detection model includes a lane line detection model and a road sign detection model; Get video stream data; Performing frame extraction processing on the video stream data to obtain corresponding road asset image data, wherein the frame extraction processing is performed by using opencv or ffmpeg; The road asset image data is input into the road asset detection model for detection to obtain a lane line point set and the coordinates, category information and confidence of road signs. The lane line point set is used to determine the lane lines, and the coordinates, category information and confidence of the road signs are used to determine the road signs. The determined lane lines and road signs are also used for motor vehicle violation / non-motor vehicle violation event detection.

2. The method according to claim 1, characterized in that After obtaining the road asset image dataset, the method further includes: performing data enhancement processing on the road asset image dataset to obtain an enhanced road asset image dataset; The road asset detection model is obtained by training using images in the enhanced road asset image dataset.

3. The method according to claim 1, characterized in that The road asset image dataset includes a lane line dataset and a road sign dataset.

4. A road asset detection device, characterized in that: include: A first acquisition module is configured to acquire a road asset image dataset, wherein the road asset image dataset includes multi-angle road sign images and multi-angle lane line images collected from multiple scenes and multiple time periods; a training module, configured to classify images in the road asset image dataset according to target attributes and input a set of labeled image samples after classification into a preset neural network for training until a road asset detection model that meets the requirements is obtained, wherein the set of labeled image samples after classification includes labeled road sign images and labeled lane line images, and the road asset detection model includes a lane line detection model and a road sign detection model; The second acquisition module is used to acquire video stream data; A second processing module is used to perform frame extraction processing on the video stream data to obtain corresponding road asset image data, wherein the frame extraction processing is performed by opencv or ffmpeg; A detection module is configured to input the road asset image data into the road asset detection model for detection, thereby obtaining a lane line point set and the coordinates, category information, and confidence levels of road signs. The lane line point set is used to determine lane lines, and the coordinates, category information, and confidence levels of road signs are used to determine road signs. The determined lane lines and road signs are also used for detecting motor vehicle / non-motor vehicle violation events.

5. The device according to claim 4, characterized in that The device further comprises: a first processing module, configured to perform data enhancement processing on the road asset image dataset to obtain an enhanced road asset image dataset; The first training module is configured to obtain the road asset detection model by training images in the enhanced road asset image dataset.

6. The device according to claim 4, characterized in that The road asset image dataset includes a lane line dataset and a road sign dataset.

7. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the road asset detection method according to any one of claims 1 to 3.

8. An electronic device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the road asset detection method according to any one of claims 1 to 3 by executing the computer instructions.

Images