Road scene recognition method, device, server, storage medium and program product

Abstract

The disclosed embodiments relate to a road scene recognition method, device, server, storage medium and program product. The method comprises: receiving location information sent by a user terminal; obtaining a scene recognition result of a target road that matches the location information; wherein the scene recognition result is used to indicate whether the target road is a road construction scene; the scene recognition result is determined based on the road feature information in the road image and the classification result of the road image; the target road is determined based on the preset planned path and location information; the scene recognition result is sent to the user terminal so that the user terminal displays the scene recognition result at a position corresponding to the target road in the electronic map. The use of this method can improve the accuracy of the recognition result indicating whether the road image is a road construction scene, so that the user can obtain a more accurate recognition result of the construction scene of the target road.

Description

Technical Field

[0001] The embodiments of the present disclosure relate to the field of intelligent transportation technology, and in particular, to a road scene recognition method, device, server, storage medium, and program product. Background Art

[0002] With the increase in social travel and the gradual improvement of urban infrastructure, construction scenes such as road construction and road maintenance are appearing more and more frequently, and road construction is often accompanied by measures such as full or partial road closures, which seriously affect user travel. Therefore, timely identification of road construction scenes can effectively improve users' travel experience, ease traffic congestion, and reduce the occurrence of traffic accidents.

[0003] In traditional technology, the recognition of road construction scenes is mainly obtained from online news, user reports, and road construction information released by government departments. Summary of the invention

[0004] The embodiments of the present disclosure provide a road scene recognition method, device, server, storage medium and program product, which can be used to improve the recognition accuracy of road construction scenes, so that users can obtain more accurate recognition results of the construction scenes of the target roads.

[0005] In a first aspect, an embodiment of the present disclosure provides a road scene recognition method, the method comprising:

[0006] Receiving location information sent by a user terminal;

[0007] Obtaining a scene recognition result of a target road that matches the location information; wherein the scene recognition result is used to indicate whether the target road is a road construction scene; the scene recognition result is determined based on road feature information in a road image and a classification result of the road image; the target road is determined based on a preset planned path and the location information;

[0008] The scene recognition result is sent to the user terminal so that the user terminal displays the scene recognition result at a position corresponding to the target road in the electronic map.

[0009] In a second aspect, an embodiment of the present disclosure provides a road scene recognition device, the device comprising:

[0010] A receiving module, used for receiving location information sent by a user terminal;

[0011] A first acquisition module is used to acquire a scene recognition result of a target road that matches the location information; wherein the scene recognition result is used to indicate whether the target road is a road construction scene; the scene recognition result is determined based on road feature information in a road image and a classification result of the road image; the target road is determined based on a preset planned path and the location information;

[0012] The sending module is used to send the scene recognition result to the user terminal, so that the user terminal displays the scene recognition result at a position corresponding to the target road in the electronic map.

[0013] In a third aspect, an embodiment of the present disclosure provides a server, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method described in the first aspect when executing the computer program.

[0014] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the method described in the first aspect above.

[0015] In a fifth aspect, an embodiment of the present disclosure provides a computer program product, including a computer program, which implements the method described in the first aspect when executed by a processor.

[0016] The road scene recognition method, device, server, storage medium and program product provided by the embodiments of the present disclosure can determine the target road according to the location information and the preset planned path by receiving the location information sent by the user terminal, so as to obtain the scene recognition result of the target road matching the location information. In this way, the obtained scene recognition result can be sent to the user terminal, so that the user terminal can display the scene recognition result at the position corresponding to the above location information in the electronic map, so that the user can obtain more accurate recognition results of the target road construction scene. In addition, the above-mentioned determined scene recognition results are determined by the server according to the road feature information in the road image and the classification result of the road image. The road feature information can mine the characteristics of the road construction scene from different angles, so as to accurately judge whether the road image is a road construction scene, thereby improving the accuracy of the obtained recognition result indicating whether the road image is a road construction scene. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 A diagram showing an application environment of a road scene recognition method in an embodiment;

[0018] Figure 2 is a schematic flow chart of a road scene recognition method in one embodiment;

[0019] Figure 3 is a schematic flow chart of a road scene recognition method in another embodiment;

[0020] Figure 4 is a schematic flow chart of a road scene recognition method in another embodiment;

[0021] Figure 5 is a schematic flow chart of a road scene recognition method in another embodiment;

[0022] Figure 6 is a schematic flow chart of a road scene recognition method in another embodiment;

[0023] Figure 7 is a structural block diagram of a road scene recognition device in one embodiment;

[0024] Figure 8 FIG. 4 is a diagram showing the internal structure of a server in one embodiment. DETAILED DESCRIPTION

[0025] In order to make the purpose, technical solution and advantages of the embodiments of the present disclosure more clear, the embodiments of the present disclosure are further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the embodiments of the present disclosure and are not used to limit the embodiments of the present disclosure.

[0026] First of all, before specifically introducing the technical solution of the embodiment of the present disclosure, the technical background or technical evolution context on which the embodiment of the present disclosure is based is introduced. Generally speaking, in the field of vehicle travel, the current technical background is: when there is road construction on the road, the construction information is mainly reported in a timely and accurate manner through online news, users, and government departments. Based on this background, the applicant has found through long-term model simulation research that when construction is carried out on some roads that are not monitored by the government or on roads that users do not often pass by, if the road construction information is not reported in a timely and accurate manner, traffic congestion and even traffic accidents will occur. How to display the construction scenes of each road to users in a timely and accurate manner has become a difficult problem that needs to be solved urgently. In addition, it should be noted that the applicant has made a lot of creative work from determining that if the road construction information is not displayed to users in a timely and accurate manner, traffic congestion and even traffic accidents will occur, as well as the technical solutions introduced in the following embodiments.

[0027] The technical solutions involved in the embodiments of the present disclosure are introduced below in combination with the scenarios to which the embodiments of the present disclosure are applied.

[0028] The road scene recognition method provided by the embodiment of the present disclosure can be applied to Figure 1In the application environment shown. Among them, the image acquisition device 102 communicates with the server 104 through the network, and the server 104 communicates with the user terminal 106 through the network. Among them, the image acquisition device 102 can be but not limited to various cameras, cameras, etc., and the image acquired by the image acquisition device can be uploaded with the authorization and consent of the user. The server 104 can be implemented with an independent server or a server cluster composed of multiple servers. Optionally, the image acquisition device 102 can be set on various vehicles, such as online-hailing cars, and can also be set on various roads. Optionally, the user terminal 106 can be a vehicle-mounted terminal device on each online-hailing car, or it can be a terminal device used by the user, such as a mobile phone, a smart watch, a tablet computer, etc.

[0029] In one embodiment, Figure 2 As shown, a road scene recognition method is provided, and the method is applied to Figure 1 The server in the example is used to illustrate the following steps:

[0030] S201: Receive location information sent by a user terminal.

[0031] The user terminal may be a vehicle-mounted terminal device on a plurality of vehicles traveling on various roads, or may be a terminal device used by a user, such as a mobile phone, a smart watch, a tablet computer, etc. Optionally, the location information sent by the user terminal may be the coordinate information of the road where the user is currently located, or may be the latitude and longitude information of the road where the user is currently located, etc. Optionally, the road where the user is currently located may be an urban road, or a rural road, or may be any of a highway, a non-motorized vehicle road, a green belt road, etc. Optionally, the user terminal may send the above location information through network communication with the server.

[0032] S202, obtaining a scene recognition result of a target road that matches the location information; wherein the scene recognition result is used to indicate whether the target road is a road construction scene; the scene recognition result is determined based on the road feature information in the road image and the classification result of the road image; the target road is determined based on a preset planned path and the location information.

[0033] Among them, the target road matching the above-mentioned location information is determined by the server according to the location information and the preset planned path. Optionally, the planned path can be sent to the server by the user terminal, or it can be a planned path related to the above-mentioned location information pre-stored in the server. The scene recognition result of the target road is used to characterize whether the target road is a road occupation construction scene, and the scene recognition result is determined by the server in advance based on the road feature information in a large number of road images obtained and the classification results of the road images. Optionally, the above-mentioned road image may include any visual elements related to the construction scene, such as people appearing on the road, roadblocks on the road, traffic signs on the road, and text signs on the road. Among them, the people appearing on the road can be construction workers on the road, or they can be policemen on the road, or they can be auxiliary policemen on the road; roadblocks on the road can include any obstacles such as cone barrels, water barriers, anti-collision barrels, fences, iron horses, etc.; traffic signs on the road can be speed limit signs on the road, or they can be lane change signs on the road. Optionally, the road feature information in the road image can include road surface element information and construction element information in the road image. The road surface element information in the road image may be road identification information on the road, for example, the road surface element information may be identification information of a no-parking zone on the road, or ground identification information on the road, or may be yellow solid line identification information on the road, or may be yellow dashed line identification information on the road, etc.; the construction element information in the road image may be information about items related to construction on the road, for example, the construction element information may be a cone barrel on the road, or a fence on the road, etc. Optionally, the classification result of the road image may be obtained by identifying the scene of the road image, and the classification result may be used to indicate whether the road image is a construction scene. Optionally, the scene recognition result pre-generated by the server may include the location information corresponding to the scene recognition result. After receiving the location information sent by the user terminal, the server may determine the target road according to the location information and the preset planned path, thereby finding the scene recognition result of the target road from the scene recognition result pre-generated by the server according to the location information of the target road. For example, the location information sent by the user terminal is the location information of road A, and the server determines that the target road is road B according to the location information of road A and the preset planned path, then the server may find the scene recognition result of road B in the pre-generated scene recognition result according to the location information of road B.

[0034] S203: Send the scene recognition result to the user terminal, so that the user terminal displays the scene recognition result at a position corresponding to the target road in the electronic map.

[0035] Optionally, the recognition result sent by the server to the user terminal may include the recognition result of whether the target road is a road-occupying construction scene. Further, if the server determines that the target road is a road-occupying construction scene, the server may also carry one or more of the time of the road-occupying construction of the target road and the location information of the target road in the scene recognition result of the target road and send it to the user terminal. Optionally, after receiving the scene recognition result, the user terminal may find the corresponding position in the electronic map according to the location information of the target road, and then display the recognition result of whether the target road is a road-occupying construction scene at the position. Further, if the scene recognition result received by the user terminal is that the target road is a road-occupying construction scene, the user terminal may also display the time information of the road-occupying construction of the target road in the map. It is understandable that after receiving the above recognition result, the user terminal may control its own display screen to display the received recognition result at the position corresponding to the above location information in the electronic map, so that the user can obtain the recognition result of whether the target road is a road-occupying construction scene in a timely manner.

[0036] In the above-mentioned road scene recognition method, the server can determine the target road according to the location information and the preset planned path by receiving the location information sent by the user terminal, so as to obtain the scene recognition result of the target road matching the location information. In this way, the obtained scene recognition result can be sent to the user terminal, so that the user terminal can display the scene recognition result at the position corresponding to the above-mentioned location information in the electronic map, so that the user can obtain more accurate recognition results of the construction scene of the target road. In addition, the above-mentioned determined scene recognition results are determined by the server according to the road feature information in the road image and the classification results of the road image. The road feature information can mine the characteristics of the road construction scene from different angles, so as to accurately judge whether the road image is a road construction scene, thereby improving the accuracy of the obtained recognition result indicating whether the road image is a road construction scene.

[0037] Further, based on the above embodiment, in one embodiment, the above S202 includes: searching a road construction information database for a scene recognition result of a target road matching the position information according to the position information.

[0038] Optionally, the server may determine the scene recognition result of each road image in advance based on the road feature information in each road image and the classification result of each road image, and store the scene recognition result of each road and the location information of each road in the road occupation construction intelligence database. When the server receives the location information sent by the user terminal, it can determine the target road based on the location information and the preset planned path, and then search in the above-mentioned road occupation construction intelligence database based on the location information of the target road to find the scene recognition result of the target road that matches the location information.

[0039] In this embodiment, the server can determine the target road that matches the location information based on the location information sent by the user terminal and the preset planned path, so that it can quickly find the scene recognition result of the target road that matches the location information from the road construction intelligence database, thereby improving the efficiency of obtaining the scene recognition result of the target road, thereby improving the efficiency of the user terminal in obtaining the scene recognition result of the target road.

[0040] In the scenario where the server searches the road occupation construction information database for a scene recognition result of a target road that matches the location information according to the location information sent by the user terminal, the server needs to predetermine the road occupation construction information database. In one embodiment, Figure 3 As shown, the above-mentioned road occupation construction intelligence database is determined based on the following steps:

[0041] S301, judging whether there is any road construction behavior in the road image according to the road feature information in the road image, and obtaining a judgment result.

[0042] Among them, the server can establish a communication protocol with multiple vehicles traveling on each road in advance or in real time through an inquiry instruction, and multiple vehicles traveling on each road are allowed to send the road images acquired by their own image acquisition devices to the server with the authorization and consent of the user, or the server can establish a communication protocol with multiple image acquisition devices set on each road in advance or in real time through an inquiry instruction, and multiple image acquisition devices set on each road can send the acquired road images to the server with the authorization and consent of the user. Optionally, the above-mentioned image acquisition device can be a camera, or a camera, or a video camera, etc. Optionally, the above-mentioned road image can be an urban road image, or a rural road image, or any road image of a highway image, a non-motorized road image, or a green belt image. Optionally, the road image can be an RGB image, or an image of other channels, which is not limited in this embodiment. It can be understood that the road image in this embodiment can be a road image or multiple road images. Furthermore, if the road image is multiple road images, the multiple road images are road images acquired on different roads respectively. Optionally, the road feature information in the road image may include road surface element information and construction element information in the road image. Optionally, the road surface element information in the road image may be identification information of a no-parking zone on the road, or ground identification information on the road, or solid yellow line identification information on the road, or yellow dashed line identification information on the road, etc.; the construction element information in the road image may be information on items related to construction on the road, for example, the construction element information may be a cone on the road, or a fence on the road, etc. Optionally, the judgment result obtained by the server may be that there is a road-occupying construction behavior in the road image, or that there is no road-occupying construction behavior in the road image. Optionally, the server may judge whether there is a road-occupying construction behavior in the road image by judging whether there is the construction element information in the road surface element information in the road image. For example, if there is a cone in the yellow dashed line identification in the road image, the server may determine that there is a road-occupying construction behavior in the road image; if there is no cone or fence in the yellow dashed line identification in the road image, the server may determine that there is no road-occupying construction behavior in the road image. Optionally, the server may obtain the road feature information in the road image from the road image through a feature extraction algorithm, or may input the road image into a pre-trained feature extraction model to obtain the road feature information in the road image.

[0043] S302, inputting the road image into a preset scene classification model to obtain a classification result; the classification result is used to indicate whether the road image is a construction scene.

[0044] It is understandable that roads in non-construction scenes often have a higher degree of traffic, and the probability of pedestrians and roadblocks appearing is relatively low. The server can pre-train the initial scene classification model through a large number of road sample images to obtain the above-mentioned scene classification model, and then use the scene classification model to obtain the classification result of the above-mentioned road image. Optionally, the road sample image may include pedestrians and roadblocks, or may not include pedestrians and roadblocks. When the road sample image includes pedestrians and roadblocks, the classification result obtained by the server using the initial scene classification model indicates that the road sample image is a construction scene; when the road sample image does not include pedestrians and roadblocks, the classification result obtained by the server using the initial scene classification model indicates that the road sample image is a non-construction scene. Optionally, the above-mentioned scene classification model can be constructed based on a convolutional neural network (CNN) or a classification network based on a Transformer. Optionally, the obtained classification result can indicate that the above-mentioned road image is a construction scene or a non-construction scene.

[0045] S303, determining a recognition result of the road image according to the judgment result and the classification result; the recognition result is used to indicate whether the road image is a road construction scene.

[0046] It can be understood that the above judgment result is used to indicate that there is a road-occupying construction behavior in the above road image, and the above classification result is used to indicate whether the above road image is a construction scene. As an achievable implementation method, the server can determine the recognition result of the road image based on one of the judgment result or the classification result. For example, if the judgment result obtained by the server indicates that there is a road-occupying construction behavior in the above road image, the server can obtain a recognition result indicating that the above road image is a road-occupying construction scene. For another example, if the classification result obtained by the server indicates that the above road image is a non-construction scene, the server can also obtain a recognition result indicating that the above road image is a road-occupying construction scene. Optionally, as another achievable implementation method, the server can combine the judgment result and the classification result to determine the recognition result of the road image. For example, when there is a road-occupying construction behavior in the road image and the road image is an image of a construction scene, the server can determine that the recognition result of the above road image indicates that the above road image is a road-occupying construction scene.

[0047] S304: If the recognition result indicates that the road is a road-occupying construction scene, the recognition result is added to the road-occupying construction information database.

[0048] Optionally, the server can directly add the obtained recognition result to the road occupation construction intelligence database, or compare the obtained recognition result with the recognition result stored in the road occupation construction intelligence database. If there is a duplicate recognition result, the recognition result can replace the existing duplicate recognition result. Optionally, in addition to the recognition result of each road image, the road occupation construction intelligence database can also include the upload time of each road image. If the recognition result obtained by the server indicates that the road is a road occupation construction scene, the server can also determine the upload time of the road image and add the upload time of the road image and the recognition result of the road image to the road occupation construction intelligence database.

[0049] In this embodiment, the server can obtain a judgment result indicating whether there is road-occupying construction behavior in the road image by judging the road feature information in the road image, and can obtain a classification result indicating whether the road image is a construction scene by inputting the road image into a preset scene classification model. Therefore, the recognition result indicating whether the road image is a road-occupying construction scene can be determined based on the obtained judgment result and classification result. By mining the characteristics of the road construction scene from different angles, it is possible to accurately judge whether the acquired road image is a road-occupying construction scene, thereby improving the accuracy of the recognition result indicating whether the road image is a road-occupying construction scene. Therefore, the recognition result indicating that the road is a road-occupying construction scene can be added to the road-occupying construction intelligence database to increase the richness of the recognition results stored in the road-occupying construction intelligence database, so that the road-occupying construction intelligence database can store more comprehensive recognition results of whether the road is a road-occupying construction scene.

[0050] Furthermore, in one embodiment, the above-mentioned road feature information is determined based on the following steps: recognizing a road image, and acquiring element information and landmark line information in the road image as the road feature information.

[0051] Optionally, the server may identify the road image through an image detection algorithm to obtain element information and landmark information in the road image, or input the road image into a pre-trained recognition model to obtain element information and landmark line information in the road image through the recognition model. The element information in the road image generally includes but is not limited to any information of people appearing on the road, roadblocks on the road, traffic signs on the road, and text signs on the road. People appearing on the road may include but are not limited to construction workers on the road, police on the road, auxiliary police on the road, etc.; roadblocks on the road may include any obstacles such as cones, water barriers, anti-collision barrels, fences, iron horses, etc.; traffic signs on the road may be speed limit signs on the road, or may be lane change signs on the road, etc.; text signs on the road may include construction signs, etc. The landmark line information in the road image can be road marking information on the road. The landmark line information in the road image can include any type of road marking information, including ground marking information on the road, solid yellow line marking information on the road, dashed yellow line marking information on the road, solid white line marking information on the road, dashed white line marking information on the road, no-parking area marking information on the road, and pedestrian crossing marking information on the road.

[0052] In this embodiment, the server can accurately obtain element information and landmark line information in the road image by identifying the road image, thereby using the obtained element information and landmark line information as road feature information to improve the accuracy of the determined road feature information. Furthermore, based on the determined road feature information with higher accuracy, it can accurately judge whether there is any road construction behavior in the obtained road image, thereby improving the accuracy of the judgment result.

[0053] Further, in one embodiment, if Figure 4 As shown, the above-mentioned road image is recognized to obtain the element information and landmark line information in the road image, including:

[0054] S401, inputting a road image into a detection network model to obtain a detection result of scattered feature information in the road image, and inputting the detection result of the scattered feature information into a recognition network model to obtain feature information.

[0055] Among them, the scattered element information in the road image may include any information such as the person information appearing in the road image, the roadblock information in the road image, the signboard information in the road image, the text signboard information in the road image, etc. It should be noted here that the detection result of the scattered element information obtained can be a detection box, etc. Further, as an optional implementation, the server can input the detection result of the scattered element information obtained, such as the detection box, into the recognition network model to classify and identify the scattered element information to obtain the element information in the road image. Optionally, the above-mentioned detection network model can be constructed based on a commonly used one-stage target detection network (such as Yolo, Retina-Net, FCOS, etc.), or a two-stage target detection network (Faster R-CNN, Cascade R-CNN) and other network models. Optionally, the above-mentioned recognition network model can be a ResNet network model, or it can also be other convolutional neural network models.

[0056] S402: Input the road image into a road feature segmentation model to obtain landmark line information in the road image.

[0057] Specifically, the pavement feature segmentation model in this embodiment is used to segment and identify the landmark line information in the road image. For example, the pavement feature segmentation model can identify the ground in the road image, the solid yellow line in the road image, the dashed yellow line in the road image, the solid white line in the road image, the dashed white line in the road image, the no-parking area in the road image, the crosswalk in the road image, and other road marking information that may appear on the road. Optionally, the pavement feature segmentation model can first segment the landmark line information in the road image, and then identify the segmented landmark line information to obtain the landmark line information in the road image. Alternatively, the pavement feature segmentation model can first identify the landmark line information in the road image, and then segment the landmark line information in the road image according to the identification result to obtain the landmark line information in the road image.

[0058] In this embodiment, the server can obtain the detection results of scattered feature information in the road image by inputting the road image into the detection network model, so that the detection results of the scattered feature information can be input into the recognition network model to accurately obtain the feature information in the road image. By inputting the road image into the pavement feature segmentation model, the landmark line information in the road image can be accurately obtained, and the obtained feature information and landmark line information with higher accuracy can be used as road feature information, thereby improving the accuracy of the determined road feature information.

[0059] Furthermore, in one embodiment, the above S303 includes: if the judgment result indicates that there is road-occupying construction behavior in the road image, and the classification result indicates that the road image is a construction scene, then determining that the recognition result indicates that the road is a road-occupying construction scene.

[0060] Specifically, in this embodiment, the server can determine whether the road image is a road-occupying construction scene in combination with the above-obtained judgment result and classification result, and obtain a recognition result indicating whether the road image is a road-occupying construction scene. It can be understood that the above judgment result is used to indicate that there is a road-occupying construction behavior in the road image, and the above classification result is used to indicate whether the road image is a construction scene. In this embodiment, if the judgment result obtained by the server indicates that there is a road-occupying construction behavior in the road image, and the above classification result indicates that the road image is a construction scene, the server can obtain a recognition result indicating that the road is a road-occupying construction scene. Optionally, if the judgment result obtained by the server indicates that there is a road-occupying construction behavior in the road image, but the classification result obtained indicates that the road image is a non-construction scene, the server can obtain a recognition result indicating that the road image is a non-road-occupying construction scene in combination with the judgment result and the classification result. For another example, if the judgment result obtained by the server indicates that there is no road-occupying construction behavior in the road image, and the above classification result indicates that the road image is a non-construction scene, the server can obtain a recognition result indicating that the road image is a non-road-occupying construction scene in combination with the judgment result and the classification result.

[0061] In this embodiment, the server determines that the recognition result of the road image indicates that the road is a road-occupying construction scene only when it determines that the judgment result indicates that there is road-occupying construction behavior in the road image and that the classification result indicates that the road image is a construction scene. By combining the judgment result and the classification result, it can accurately determine whether the road image is a road-occupying construction scene, thereby improving the accuracy of the determined recognition result.

[0062] In some scenarios, if the recognition result obtained by the server indicates that the road is a road construction scene, the server may also compare the recognition result of the road image obtained with the information in the road construction situation database to determine whether the recognition result of the road image obtained is redundant information, and add the recognition result of the road image to the road construction information database according to the determination result. Based on the above embodiment, in one embodiment, if Figure 5 As shown, the above S304 includes:

[0063] S501, determining the information of the road image; the information of the road image includes the uploading time and uploading location of the road image.

[0064] Specifically, if the server determines that the recognition result of the road image indicates that the road is a road construction scene, the server can determine the information of the road image according to the acquisition time of the road image, and determine the upload location of the road image according to the location information of the road image. For example, if the server receives the uploaded road image in real time, the server can determine the current time of acquiring the road image as the upload time of the road image; if the road image is received in advance by the server, the server can determine the time of acquiring the road image as the upload time of the road image. Optionally, the server can receive the upload location of the road image, alternatively or additionally, the server can determine the upload location of the road image based on the landmark elements in the uploaded road image.

[0065] S502, comparing the information of the road image with the information in the road occupation construction information database to determine whether there is road occupation information of the road in the road occupation construction information database.

[0066] Specifically, the server compares the upload time of the road image with the release time of the road occupation construction information in the road occupation construction information database, and compares the upload location of the road image with the location information of the road occupation construction information in the road occupation construction information database, to determine whether the road occupation information of the road in the road image exists in the road occupation construction information database. Optionally, if the release time of the road occupation construction information in the road occupation construction information database is consistent with the upload time of the road image, and the location information of the road occupation construction information in the road occupation construction information database is consistent with the upload location of the road image, then the server can determine that the road occupation information of the road in the road image exists in the road occupation construction information database. Optionally, if the release time of the road occupation construction information in the road occupation construction information database is inconsistent with the upload time of the road image, or the location information of the road occupation construction information in the road occupation construction information database is inconsistent with the upload location of the road image, then the server can determine that the road occupation information of the road in the road image does not exist in the road occupation construction information database. Further, if the server determines that the road occupation information of the road in the above road image does not exist in the road occupation construction information database, step S503 can be executed. If the server determines that the road occupation information of the road in the above road image exists in the road occupation construction information database, step S504 can be executed.

[0067] S503, adding the identification result to the road occupation construction information database.

[0068] Specifically, if the server determines that the road occupation information of the road represented by the recognition result of the above road image does not exist in the above road occupation information database, the server can add the recognition result of the above road image to the road occupation information database and update the information in the road occupation information database.

[0069] S504, replacing the road occupation information of the road in the road occupation construction information database with the recognition result.

[0070] Specifically, if the server determines that the road occupation information of the road represented by the recognition result of the above road image exists in the above road occupation information database, the server can replace the road occupation information of the road in the road occupation information database with the above recognition result to update the information in the road occupation information database.

[0071] In this embodiment, when the server determines that the recognition result of the acquired road image indicates that the road is under road occupation construction, the server determines the upload time and upload location of the road image, compares the upload time and upload location of the road image with the information in the road occupation construction intelligence database, and when it is determined that the road occupation intelligence information of the road does not exist in the road occupation construction intelligence database, the recognition result of the road image is added to the road occupation construction intelligence database; when it is determined that the road occupation intelligence information of the road exists in the road occupation construction intelligence database, the recognition result of the road image is used to replace the road occupation intelligence information of the road in the road occupation construction situation database, thereby avoiding the situation where redundant information exists in the road occupation construction intelligence database and ensuring the accuracy of the intelligence information in the road occupation construction intelligence database.

[0072] Before adding the recognition result of the road image to the road construction situation database, the server may also determine the validity of the recognition result of the road image obtained by obtaining the result of manual review of whether the road image is a road construction scene. If the recognition result of the road image is valid, the recognition result is added to the road construction situation database. Based on the above embodiment, in one embodiment, Figure 6 As shown, the above-mentioned road occupation construction intelligence database is updated based on the following steps:

[0073] S601, obtaining a review instruction; the review instruction includes a review result of the manual operation platform on whether the road image is a road occupation construction scene.

[0074] Specifically, the server can obtain the review instruction sent by the manual operation platform through network communication with the manual operation platform, and the review instruction can include the review result of the manual operation platform on whether the road image is a road-occupying construction scene. Optionally, the review result of the manual operation platform on the road image can be that the road image is a road-occupying construction scene, or that the road image is not a road-occupying construction scene.

[0075] S602: judging the validity of the identification result according to the audit result.

[0076] Specifically, the server judges the validity of the recognition result of the road image obtained above based on the audit result of whether the above road image is a road construction scene obtained by the manual operation platform, thereby obtaining a validity judgment result of the above recognition result. Optionally, the validity judgment result of the above recognition result obtained by the server may be that the above recognition result is valid, or may be that the above recognition result is invalid. It can be understood that if the validity judgment result of the above recognition result obtained by the server is that the above recognition result is valid, it means that the recognition result of the above road image determined by the server is accurate and the recognition result of the above road image can be used; if the validity judgment result of the above recognition result obtained by the server is that the above recognition result is invalid, it means that the recognition result of the above road image determined by the server is incorrect.

[0077] S603, when the recognition result is valid, execute the step of adding the recognition result to the road occupation construction information database.

[0078] Specifically, when the server determines that the recognition result of the road image obtained above is valid, the server executes the above step S305. That is to say, when the server determines that the recognition result of the road image obtained above is valid, the server adds the recognition result of the road image obtained to the above road occupation construction information database, and updates the recognition result of the road image in the road occupation construction information database.

[0079] In this embodiment, the server obtains an audit instruction including the audit result of the manual work platform on whether the road image is a road occupying construction scene. The server can judge the validity of the recognition result of the road image based on the audit result of the manual work platform on whether the road image is a road occupying construction scene. Therefore, if it is determined that the recognition result of the road image is valid, the recognition result of the road image can be added to the road occupying construction intelligence database, further ensuring the accuracy of the intelligence data added to the road occupying construction intelligence database.

[0080] An embodiment of the present disclosure is introduced below in conjunction with a specific travel scenario. The method includes the following steps:

[0081] S1, establish a communication protocol with multiple vehicles traveling on each road in advance or in real time through an inquiry instruction, and obtain road images acquired by image acquisition devices of multiple vehicles traveling on each road with the authorization and consent of the user; or; establish a communication protocol with multiple image acquisition devices set on each road in advance or in real time through an inquiry instruction, and acquire road images acquired by multiple image acquisition devices set on each road.

[0082] S2, inputting the road image into the detection network model to obtain the detection result of the scattered feature information in the road image.

[0083] S3, inputting the detection results of the scattered feature information into the recognition network model to obtain the feature information.

[0084] S4, inputting the road image into the road feature segmentation model to obtain the landmark line information in the road image.

[0085] S5, taking the element information and landmark line information in the road image as road feature information.

[0086] S6, judging whether there is any road construction behavior in the road image according to the road feature information in the road image, and obtaining a judgment result.

[0087] S7, inputting the road image into a preset scene classification model to obtain a classification result; the classification result is used to indicate whether the road image is a construction scene.

[0088] S8, if the judgment result indicates that there is road-occupying construction behavior in the road image, and the classification result indicates that the road image is a construction scene, then determine that the recognition result indicates that the road is a road-occupying construction scene.

[0089] S9, determining the information of the road image; the information of the road image includes the upload time and upload location of the road image.

[0090] S10, comparing the information of the road image with the information in the road occupation construction information database to determine whether there is road occupation information of the road in the road occupation construction information database.

[0091] S11, if the road occupation intelligence information of the road does not exist in the road occupation construction intelligence database, obtain the review instruction including the review result of the manual operation platform on whether the road image is a road occupation construction scene, judge the validity of the recognition result according to the review result, and add the recognition result to the road occupation construction intelligence database if the recognition result is valid.

[0092] S12, if the road occupation information of the road exists in the road occupation construction information database, the road occupation information of the road in the road occupation construction information database is replaced with the recognition result.

[0093] S13, receiving location information sent by the user terminal.

[0094] S14, determining a target road matching the above position information according to the preset planned path and position information.

[0095] S15, obtaining a scene recognition result of the target road that matches the location information from a road occupation construction information database; the scene recognition result is used to indicate whether the target road is a road occupation construction scene.

[0096] S16, sending the scene recognition result to the user terminal, so that the user terminal displays the scene recognition result at a position corresponding to the target road in the electronic map.

[0097] For the working principle of the road scene recognition method provided in this embodiment, please refer to the detailed description in the above embodiment, which will not be described in detail in this embodiment.

[0098] It should be understood that although Figure 2-Figure 6 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 2-Figure 6 At least part of the steps may include multiple steps or multiple stages. These steps or stages are not necessarily performed at the same time, but can be performed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed in turn or alternately with other steps or at least part of the steps or stages in other steps.

[0099] In one embodiment, Figure 7 As shown, a road scene recognition device is provided, comprising: a receiving module, a first acquisition module and a sending module, wherein:

[0100] The receiving module is used to receive the location information sent by the user terminal.

[0101] The first acquisition module is used to obtain the scene recognition result of the target road that matches the location information; wherein the scene recognition result is used to indicate whether the target road is a road construction scene; the scene recognition result is determined based on the road feature information in the road image and the classification result of the road image; the target road is determined based on the preset planned path and location information.

[0102] The sending module is used to send the scene recognition result to the user terminal so that the user terminal displays the scene recognition result at a position corresponding to the target road in the electronic map.

[0103] The road scene recognition device provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, which will not be repeated here.

[0104] Based on the above embodiment, optionally, the above first acquisition module includes: a first acquisition unit, wherein:

[0105] The first acquisition unit is used to search the road occupation construction intelligence database for a scene recognition result of a target road that matches the location information according to the location information.

[0106] The road scene recognition device provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, which will not be repeated here.

[0107] On the basis of the above embodiment, optionally, the above device further includes: a first judgment module, a second acquisition module, a determination module and an addition module, wherein:

[0108] The first judgment module is used to judge whether there is road occupation construction behavior in the road image according to the road feature information in the road image, and obtain a judgment result.

[0109] The second acquisition module is used to input the road image into a preset scene classification model to obtain a classification result; the classification result is used to indicate whether the road image is a construction scene.

[0110] The determination module is used to determine the recognition result of the road image according to the judgment result and the classification result; the recognition result is used to indicate whether the road image is a road construction scene.

[0111] The adding module is used to add the recognition result to the road occupation construction intelligence database if the recognition result indicates that the road is a road occupation construction scene.

[0112] The road scene recognition device provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, which will not be repeated here.

[0113] Based on the above embodiment, optionally, the above device further includes: a third acquisition module, wherein:

[0114] The third acquisition module is used to recognize the road image and acquire element information and landmark line information in the road image as road feature information.

[0115] The road scene recognition device provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, which will not be repeated here.

[0116] On the basis of the above embodiment, optionally, the above third acquisition module includes: a second acquisition unit and a third acquisition unit, wherein:

[0117] The second acquisition unit is used to input the road image into the detection network model to obtain the detection result of the scattered element information in the road image, and input the detection result of the scattered element information into the recognition network model to obtain the element information.

[0118] The third acquisition unit is used to input the road image into the road feature segmentation model to obtain the landmark line information in the road image.

[0119] The road scene recognition device provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, which will not be repeated here.

[0120] Based on the above embodiment, optionally, the above determination module includes: a first determination unit, wherein:

[0121] The first determination unit is used to determine that the recognition result indicates that the road is a road construction scene if the judgment result indicates that there is road construction in the road image and the classification result indicates that the road image is a construction scene.

[0122] The road scene recognition device provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, which will not be repeated here.

[0123] On the basis of the above embodiment, optionally, the above adding module includes: a second determining unit, a comparing unit, an adding unit and a replacing unit, wherein:

[0124] The second determining unit is used to determine the information of the road image; the information of the road image includes the uploading time and uploading location of the road image.

[0125] The comparison unit is used to compare the information of the road image with the information in the road occupation construction information database to determine whether there is road occupation information of the road in the road occupation construction information database.

[0126] The adding unit is used to add the recognition result to the road occupation construction intelligence database if the road occupation intelligence information of the road does not exist in the road occupation construction intelligence database.

[0127] The replacement module is used to replace the road occupation intelligence information of the road in the road occupation construction intelligence database with the identification result if the road occupation intelligence information of the road exists in the road occupation construction intelligence database.

[0128] The road scene recognition device provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, which will not be repeated here.

[0129] On the basis of the above embodiment, optionally, the above device further includes: a fourth acquisition module, a second judgment module and an execution module, wherein:

[0130] The fifth acquisition module is used to obtain the review instruction; the review instruction includes the review result of the manual operation platform on whether the road image is a road construction scene.

[0131] The second judgment module is used to judge the validity of the recognition result according to the audit result.

[0132] The execution module is used to add the recognition result to the road occupation construction intelligence database when the recognition result is valid.

[0133] The road scene recognition device provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, which will not be repeated here.

[0134] For the specific definition of the road scene recognition device, please refer to the definition of the road scene recognition method above, which will not be repeated here. Each module in the above-mentioned road scene recognition device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the server in the form of hardware, or can be stored in the memory of the electronic device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0135] Figure 8 1 is a block diagram of a server 1400 according to an exemplary embodiment. Figure 8 The server 1400 includes a processing component 1420, which further includes one or more processors, and a memory resource represented by a memory 1422 for storing instructions or computer programs, such as applications, that can be executed by the processing component 1420. The application stored in the memory 1422 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 1420 is configured to execute instructions to perform the above-mentioned road scene recognition method.

[0136] The server 1400 may also include a power supply component 1424 configured to perform power management of the device 1400, a wired or wireless network interface 1426 configured to connect the device 1400 to a network, and an input / output (I / O) interface 1428. The server 1400 may operate based on an operating system stored in the memory 1422, such as Windows 14 14erverTM, Mac O14 XTM, UnixTM, LinuxTM, FreeB14DTM or the like.

[0137] In an exemplary embodiment, a storage medium including instructions is also provided, such as a memory 1422 including instructions, and the instructions can be executed by a processor of the server 1400 to complete the above method. The storage medium can be a non-transitory computer-readable storage medium, for example, a non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0138] In an exemplary embodiment, a computer program product is also provided, and when the computer program is executed by a processor, the above method can be implemented. The computer program product includes one or more computer instructions. When these computer instructions are loaded and executed on a computer, part or all of the above method can be implemented in whole or in part according to the process or function of the embodiment of the present disclosure.

[0139] Exemplarily, the present application embodiment discloses TS1, a road scene recognition method, characterized in that the method includes:

[0140] Receiving location information sent by a user terminal;

[0141] Obtaining a scene recognition result of a target road that matches the location information; wherein the scene recognition result is used to indicate whether the target road is a road construction scene; the scene recognition result is determined based on road feature information in a road image and a classification result of the road image; the target road is determined based on a preset planned path and the location information;

[0142] The scene recognition result is sent to the user terminal so that the user terminal displays the scene recognition result at a position corresponding to the target road in the electronic map.

[0143] TS2. The method of TS1, wherein obtaining a scene recognition result of a target road matching the location information comprises:

[0144] According to the location information, a scene recognition result of a target road matching the location information is searched in a road occupation construction intelligence database.

[0145] TS3. In the method of TS2, the road occupation construction intelligence database is determined based on the following steps:

[0146] According to the road feature information in the road image, determining whether there is road construction in the road image, and obtaining a determination result;

[0147] Inputting the road image into a preset scene classification model to obtain a classification result; the classification result is used to indicate whether the road image is a construction scene;

[0148] Determining a recognition result of the road image according to the judgment result and the classification result; the recognition result is used to indicate whether the road image is a road construction scene;

[0149] If the recognition result indicates that the road is a road-occupying construction scene, the recognition result is added to the road-occupying construction information database.

[0150] TS4. The method of TS3, wherein the road characteristic information is determined based on the following steps:

[0151] The road image is recognized, and element information and landmark line information in the road image are obtained as the road feature information.

[0152] TS5. The method of TS4, wherein the step of identifying the road image and obtaining element information and landmark line information in the road image comprises:

[0153] Inputting the road image into the detection network model to obtain a detection result of scattered element information in the road image, and inputting the detection result of the scattered element information into the recognition network model to obtain the element information;

[0154] as well as,

[0155] The road image is input into a road feature segmentation model to obtain landmark line information in the road image.

[0156] TS6. The method according to TS3, wherein determining the recognition result of the road image according to the judgment result and the classification result comprises:

[0157] If the judgment result indicates that there is road-occupying construction behavior in the road image, and the classification result indicates that the road image is a construction scene, it is determined that the recognition result indicates that the road is a road-occupying construction scene.

[0158] TS7. The method as described in TS3, wherein if the recognition result indicates that the road is a road occupation construction scene, the recognition result is added to the road occupation construction intelligence database, including:

[0159] Determine information of the road image; the information of the road image includes the upload time and upload location of the road image;

[0160] Comparing the information of the road image with the information in the road occupation construction information database to determine whether the road occupation information of the road exists in the road occupation construction information database;

[0161] If not, the identification result is added to the road occupation construction information database;

[0162] If so, the road occupation information of the road in the road occupation construction information database is replaced by the identification result.

[0163] TS8. As described in TS3, the road occupation construction intelligence database is updated based on the following steps:

[0164] Obtaining review instructions; the review instructions include the review result of the manual operation platform on whether the road image is a road occupation construction scene;

[0165] Determine the validity of the identification results based on the audit results;

[0166] When the identification result is valid, the step of adding the identification result to the road occupation construction information database is performed.

[0167] TS9. A road scene recognition device, characterized in that the device comprises:

[0168] A receiving module, used for receiving location information sent by a user terminal;

[0169] A first acquisition module is used to acquire a scene recognition result of a target road that matches the location information; wherein the scene recognition result is used to indicate whether the target road is a road construction scene; the scene recognition result is determined based on road feature information in a road image and a classification result of the road image; the target road is determined based on a preset planned path and the location information;

[0170] The sending module is used to send the scene recognition result to the user terminal, so that the user terminal displays the scene recognition result at a position corresponding to the target road in the electronic map.

[0171] TS10. A server comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of any one of the methods in TS1 to TS8 when executing the computer program.

[0172] TS11. A storage medium having a computer program stored thereon, characterized in that when the computer program is executed by a processor, the steps of any one of the methods in TS1 to TS8 are implemented.

[0173] TS12. A computer program product, comprising a computer program, characterized in that when the computer program is executed by a processor, the steps of any one of the methods of TS1 to TS8 are implemented.

[0174] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in the embodiments of the present disclosure can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0175] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0176] The above-described embodiments only express several implementation methods of the embodiments of the present disclosure, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for those of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the embodiments of the present disclosure, and these all belong to the protection scope of the embodiments of the present disclosure. Therefore, the protection scope of the patent of the embodiments of the present disclosure shall be subject to the attached claims.

Claims

1. A road scene recognition method, characterized in that: The method comprises: Receiving location information sent by a user terminal; Obtaining a scene recognition result of a target road that matches the location information; wherein the scene recognition result is used to indicate whether the target road is a road construction scene; the scene recognition result is determined from scene recognition results determined in advance based on road feature information in a road image and a classification result of the road image; the target road is determined based on a preset planned path and the location information; the road feature information includes element information and landmark line information in the road image; The scene recognition result is sent to the user terminal so that the user terminal displays the scene recognition result at a position corresponding to the target road in the electronic map.

2. The method according to claim 1, characterized in that The obtaining of a scene recognition result of a target road matching the position information includes: According to the location information, a scene recognition result of a target road matching the location information is searched in a road occupation construction intelligence database.

3. The method according to claim 2, characterized in that The road occupation construction intelligence database is determined based on the following steps: According to the road feature information in the road image, determining whether there is road construction in the road image, and obtaining a determination result; Inputting the road image into a preset scene classification model to obtain a classification result; the classification result is used to indicate whether the road image is a construction scene; Determining a recognition result of the road image according to the judgment result and the classification result; the recognition result is used to indicate whether the road image is a road construction scene; If the recognition result indicates that the road is a road-occupying construction scene, the recognition result is added to the road-occupying construction information database.

4. The method according to claim 3, characterized in that The road characteristic information is determined based on the following steps: The road image is recognized, and element information and landmark line information in the road image are obtained as the road feature information.

5. The method according to claim 4, characterized in that The step of identifying the road image and acquiring element information and landmark line information in the road image includes: Inputting the road image into a detection network model to obtain a detection result of scattered element information in the road image, and inputting the detection result of the scattered element information into a recognition network model to obtain the element information; as well as, The road image is input into a road feature segmentation model to obtain landmark line information in the road image.

6. The method according to claim 3, characterized in that Determining the recognition result of the road image according to the judgment result and the classification result includes: If the judgment result indicates that there is road-occupying construction behavior in the road image, and the classification result indicates that the road image is a construction scene, it is determined that the recognition result indicates that the road is a road-occupying construction scene.

7. The method according to claim 3, characterized in that If the recognition result indicates that the road is a road occupation construction scene, adding the recognition result to the road occupation construction intelligence database includes: Determine information of the road image; the information of the road image includes the upload time and upload location of the road image; Comparing the information of the road image with the information in the road occupation construction information database to determine whether the road occupation information of the road exists in the road occupation construction information database; If not, the identification result is added to the road occupation construction information database; If so, the road occupation information of the road in the road occupation construction information database is replaced by the identification result.

8. The method according to claim 3, characterized in that The road occupation construction intelligence database is updated based on the following steps: Acquire a review instruction; the review instruction includes a review result of the manual operation platform on whether the road image is a road occupation construction scene; Judging the validity of the identification result according to the audit result; When the recognition result is valid, the step of adding the recognition result to the road construction information database is performed.

9. A road scene recognition device, characterized in that: The device comprises: A receiving module, used for receiving location information sent by a user terminal; A first acquisition module is used to acquire a scene recognition result of a target road that matches the location information; wherein the scene recognition result is used to indicate whether the target road is a road construction scene; the scene recognition result is determined from scene device results determined in advance based on road feature information in a road image and a classification result of the road image; the target road is determined based on a preset planned path and the location information; the road feature information includes element information and landmark line information in the road image; The sending module is used to send the scene recognition result to the user terminal, so that the user terminal displays the scene recognition result at a position corresponding to the target road in the electronic map.

10. A server comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 8 are implemented.

11. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

12. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images