Method, device and terminal equipment for determining road category
Through the combination method of two-layer deep learning models, the problem of low accuracy and credibility of the navigation system when judging road types is solved, accurate identification of road categories is achieved, and judgment accuracy is improved.
Patent Information
- Application Number
- CN202110383102.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-04-09
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2041-04-09
AI Technical Summary
Existing navigation systems have problems of low accuracy and low credibility when judging road types, especially due to insufficient jump points and accuracy in systems such as GPS.
Using the combination method of two-layer deep learning models, firstly, the road and signs are initially identified through the first deep learning model, the road situation and sign information are obtained, and the recognition results and images are input into the second deep learning model, and the road is carried out through confidence. Precise identification of categories.
Accurate identification of road categories is achieved, judgment accuracy is improved, and road types can be accurately judged when road forks are crossed, avoiding entering wrong road forks.
Smart Images

Figure CN113221652B_ABST
Abstract
Description
Technical Field
[0001] The present application belongs to the field of intelligent identification technology, and in particular, relates to a method, apparatus, terminal device and computer-readable storage medium for determining a road category. Background Art
[0002] At present, when a vehicle is driving, it usually uses navigation equipment to determine the type of road it is traveling on, for example, whether the road it is traveling on is a main road or a secondary road. However, since navigation systems such as the Global Positioning System (GPS) may have problems such as skipping points and insufficient accuracy, the accuracy of determining the type of road the vehicle is traveling on is low and the credibility is low. Summary of the invention
[0003] The embodiments of the present application provide a method for determining a road category, an apparatus for determining a road category, a terminal device, and a computer-readable storage medium, which can more accurately determine the type of road.
[0004] In a first aspect, an embodiment of the present application provides a method for determining a road category, comprising:
[0005] Inputting the first image into a first deep learning model to obtain a first recognition result for the first image, wherein the first recognition result includes a road recognition result and a sign recognition result in the first image;
[0006] Inputting the first recognition result and the first image into a second deep learning model to obtain a second recognition result, wherein the second recognition result includes a confidence level of a category of a road recognized in the first image;
[0007] According to the second recognition result, a road category in the first image is determined.
[0008] In a second aspect, an embodiment of the present application provides a device for determining a road category, including:
[0009] A first recognition module, configured to input a first image into a first deep learning model to obtain a first recognition result for the first image, wherein the first recognition result includes a road recognition result and a sign recognition result in the first image;
[0010] A second recognition module, configured to input the first recognition result and the first image into a second deep learning model to obtain a second recognition result, wherein the second recognition result includes a confidence level of a category of a road recognized in the first image;
[0011] A determination module is used to determine the road category in the first image according to the second recognition result.
[0012] In a third aspect, an embodiment of the present application provides a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method for determining the road category as described in the first aspect when executing the computer program.
[0013] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for determining the road category as described in the first aspect is implemented.
[0014] In a fifth aspect, an embodiment of the present application provides a computer program product, which, when executed on a terminal device, enables the terminal device to execute the method for determining the road category described in the first aspect above.
[0015] Compared with the prior art, the embodiments of the present application have the following beneficial effects: in the embodiments of the present application, a first recognition result for the first image can be obtained through a first deep learning model, so that the road conditions and sign conditions in the first image can be identified, thereby realizing a preliminary perception of the road and related environment in the first image through the first deep learning model. Then, the first recognition result and the first image are input into a second deep learning model to obtain a second recognition result, and the second recognition result includes the confidence of the category of the road identified in the first image; at this time, the category of the road in the first image can be more accurately identified through the second deep learning model in combination with the preliminary perception of the road and related environment in the first image by the first deep learning model, and a second recognition result can be obtained, so that the category of the road in the first image can be more accurately determined based on the second recognition result. BRIEF DESCRIPTION OF THE DRAWINGS
[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0017] Figure 1 is a flowchart of a method for determining a road category provided in an embodiment of the present application;
[0018] Figure 2 is an exemplary schematic diagram of a road surface recognition result provided by an embodiment of the present application;
[0019] Figure 3is an exemplary schematic diagram of a logo recognition result provided by an embodiment of the present application;
[0020] Figure 4 is a schematic diagram of the structure of a device for determining a road category provided by an embodiment of the present application;
[0021] Figure 5 It is a structural diagram of a terminal device provided in one embodiment of the present application. DETAILED DESCRIPTION
[0022] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.
[0023] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.
[0024] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.
[0025] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.
[0026] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.
[0027] Specifically, Figure 1 A flowchart of a method for determining a road category provided in an embodiment of the present application is shown. The method for determining a road category can be applied to a terminal device.
[0028] Exemplarily, the terminal device may be a server, a desktop computer, a mobile phone, a tablet computer, a wearable device, an in-vehicle device, an augmented reality (AR) / virtual reality (VR) device, a laptop computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), etc. The embodiments of the present application do not impose any restrictions on the specific type of the terminal device.
[0029] In one embodiment, the terminal device may be deployed on a designated vehicle, so as to conveniently prompt the driver of the designated vehicle with the current road category so that the driver can drive on the correct road.
[0030] In another embodiment, the terminal device may also be a cloud server. In this case, the computing power of the terminal device is higher than that of the device deployed on the designated vehicle, so that more complex and accurate calculations can be performed. In a specific scenario, the cloud server can obtain the first image captured by the camera on the designated vehicle through a specific information transmission method, thereby executing the embodiment of the present application. After the cloud server determines the road category in the first image, the recognition result can be sent to the on-board device of the designated vehicle.
[0031] Specifically, Figure 1 As shown, the method for determining the road category may include:
[0032] Step S101: input a first image into a first deep learning model to obtain a first recognition result for the first image, where the first recognition result includes a road recognition result and a sign recognition result in the first image.
[0033] In an embodiment of the present application, the first image may be an image containing the front driving scene information of the specified vehicle. The number and format of the first images are not limited here and can be determined according to specific scene requirements, the specific type and requirements of the first deep learning model, etc.
[0034] For example, in one example, the first image includes N consecutive video frames in the target video, where N is an integer not less than 2. At this time, each of the video frames can be input into the first deep learning model to obtain a first recognition result for each of the video frames. In the subsequent processing, the first recognition result and the corresponding video frame can be input into the second deep learning model for each first recognition result to obtain a second recognition result of the video frame corresponding to the first recognition result, thereby determining the road category based on the second recognition results corresponding to each video frame. At this time, the recognition results of the road categories in the continuous video frames can be used to finally obtain a road category with a higher confidence as the final recognition result, thereby improving the recognition accuracy. Of course, in some other examples, the first image can also be an image.
[0035] There are many specific ways to obtain the first image. For example, in some examples, after the camera on the designated vehicle acquires the first image, the first image is transmitted to the terminal device executing the embodiment of the present application, or the first image is pre-stored in the terminal device. The first image can be an image captured by a camera, or a video frame in a video, or a preview image of a camera, etc.
[0036] The first deep learning model may be a target detection model, so as to detect roads and signs in the first image. Exemplarily, the first deep learning model may be a Faster R-CNN model, an SSD model, or a YOLO model. The specific structure and parameter settings of the first deep learning model may be set according to specific scene requirements and are not limited here.
[0037] The first deep learning model may be a trained deep learning model. Specifically, the first deep learning model may be obtained by training an initial first deep learning model with a training image containing a road label and a training image containing a logo label. When training the initial first deep learning model, whether the training is completed may be determined by a specified loss function. For example, if the output result of a certain iteration process in the training process satisfies the first loss condition based on the specified loss function, or the training reaches a preset maximum number of iterations, the training may be considered completed and the first deep learning model may be obtained.
[0038] The loss function corresponding to the first deep learning model is not limited here. Exemplarily, the loss function corresponding to the first deep learning model can be a loss function based on mean square error or a loss function based on cross entropy, etc.
[0039] The road recognition result may include the roads and non-roads identified in the first image. Among them, the roads in the embodiment of the present application may include various lanes, and the non-roads may be some impassable roads and objects such as isolation belts, bridge piers, street lights, etc. The sign recognition result may include the road signs identified in the first image, such as road lines, indicator markings on the road (such as yield arrows, straight arrows, left turn arrows, right turn arrows, etc.). In addition, the sign recognition result may also include non-road signs, such as green belt signs, shoulders, street lights, etc.
[0040] The road and related environment conditions in the first image can be preliminarily characterized by the road recognition results and the sign recognition results, so that they can be used to more accurately recognize the road conditions in the first image in the subsequent process.
[0041] In some embodiments, before inputting the first image into the first deep learning model to obtain a first recognition result for the first image, the method further includes:
[0042] Get the target video captured by the camera on the specified vehicle;
[0043] The first image is determined from the target video, wherein the first image contains the front driving scene information of the designated vehicle.
[0044] In an embodiment of the present application, the driving scene in front of the designated vehicle can be photographed by a camera on the designated vehicle, thereby helping the driver and the navigation system to accurately determine the type of road the designated vehicle is currently on. Moreover, when driving to a fork in the road, the road types of the various roads in front of the designated vehicle can also be determined to avoid entering the wrong fork in the road.
[0045] In some embodiments, the first deep learning model includes a first deep learning sub-model and a second deep learning sub-model;
[0046] The step of inputting the first image into a first deep learning model to obtain a first recognition result for the first image includes:
[0047] Inputting the first image into the first deep learning sub-model to obtain a road surface recognition result in the first image;
[0048] The road surface recognition result and the first image are input into the second deep learning sub-model to obtain a first recognition result for the first image.
[0049] In an embodiment of the present application, the first deep learning sub-model and the second deep learning sub-model can be target detection models respectively, and are used to detect different features. The first deep learning sub-model is used to identify road surfaces and non-road surfaces in the first image. The road surface may include surfaces of various different types of roads and traffic areas.
[0050] Based on the road surface recognition result, the second deep learning sub-model can further identify the road recognition result and the sign recognition result in the first image in a more refined manner based on the road surface recognition result, wherein the road can be considered as a part of the road surface, and each sign in the sign recognition result can be based on further feature detection of the road surface and non-road surface. At this time, based on the road surface recognition result, the difficulty of identifying roads and features by the second deep learning sub-model is greatly reduced, and at the same time, a good recognition accuracy can be guaranteed.
[0051] like Figure 2 The figure shows an exemplary representation of the road surface recognition result.
[0052] Among them, Figure 2 As shown in the portion framed by the white lines, the road surface may include a main road, a secondary road, a parking area, and a transition road area between the main road and the secondary road.
[0053] like Figure 3 The figure shows an exemplary representation of the identification result. The white line with an arrow is the identified road line. The identification result may include the identification result of the road line, so that the road division can be better identified through the road line.
[0054] In some embodiments, the identification result includes a road line identification result and an auxiliary identification result. The road line identification result includes a road line identified by a road line identification code, and the auxiliary identification result includes an auxiliary identification obtained by an auxiliary identification code. In addition, in the first identification result, the road line identification code and the auxiliary identification code are arranged in a specified order, and the specified order is determined according to the position information of the road line and the auxiliary identification in the first image respectively.
[0055] The road line recognition result can be used to assist in identifying the way in which the road is divided, and the auxiliary identification result can be used to assist in determining the current road and the surrounding conditions.
[0056] Exemplarily, the auxiliary signs may include at least one of road signs, road indication signs (such as yield arrows, straight arrows, left turn arrows, right turn arrows, etc.) and non-road signs (such as isolation belt signs, green belt signs).
[0057] In an embodiment of the present application, in the first recognition result, the road line identification code and the auxiliary identification code are arranged according to a specified order, and the specified order is determined according to the position information of the road line and the auxiliary identification in the first image respectively. At this time, the first identification result can be used to preliminarily see the distribution of each feature in the first image, thereby preliminarily judging the division of roads and non-roads in the first image.
[0058] For example, in one example, a road line identification code may be set for each road line, specifically, the n road lines are respectively: L1 ... Ln. Also, an auxiliary identification code may be set for each non-road sign, specifically, the m non-road signs are respectively: P1 ... Pm.
[0059] The road line identification code and the auxiliary identification code are arranged in a specified order to obtain: P1L1L2...L5P2P3L6...LnP4...Pm. It can be seen that L1...L5 are continuous road line identification codes and are likely to be on the same road. L6...Ln are also continuous road line identification codes and are likely to be on the same road. However, L1...L5 and L6...Ln are separated by P2P3, so it can be preliminarily determined that L1...L5 and L6...Ln are located on two different roads. It can be seen that the setting method of this first recognition result can facilitate intuitively reflecting the distribution of each feature in the first image, thereby preliminarily determining the division of roads and non-roads in the first image.
[0060] Step S102: input the first recognition result and the first image into a second deep learning model to obtain a second recognition result, wherein the second recognition result includes the confidence of the category of the road recognized in the first image.
[0061] In an embodiment of the present application, the second deep learning model may be a target detection model, so as to detect the category of the road in the first image. Exemplarily, the second deep learning model may be a Faster R-CNN model, an SSD model, or a YOLO model, etc. The specific structure and parameter settings of the second deep learning model may be set according to the specific scene requirements and are not limited here.
[0062] In some examples, the second deep learning model can realize the recognition of multiple different types of roads through a multi-layer model structure. In other examples, the second deep learning model can include multiple parallel sub-models to respectively recognize different types of roads, for example, one sub-model is used to recognize a main road, and another sub-model is used to recognize a secondary road.
[0063] The second deep learning model may be a trained deep learning model. Specifically, the second deep learning model may be obtained by training the initial second deep learning model with training images containing road type labels. When training the initial second deep learning model, the completion of the training may be determined by a specified loss function. For example, if the output result of a certain iteration process in the training process satisfies the second loss condition based on the specified loss function, or the training reaches a preset maximum number of iterations, the training may be considered completed and the second deep learning model may be obtained.
[0064] The loss function corresponding to the second deep learning model is not limited here. For example, the loss function corresponding to the second deep learning model can be a loss function based on mean square error or a loss function based on cross entropy, etc.
[0065] The structure of the second deep learning model may be the same as or different from that of the first deep learning model.
[0066] In the embodiment of the present application, the road category can be set according to the actual scene requirements. For example, the road category can include a main road and a secondary road, or the road category can also include a fast lane, a slow lane, a bus-only lane, etc.
[0067] The second recognition result may represent the confidence level of the road category recognized in the first image. For example, the second recognition result may include the following information:
[0068] The confidence level of the primary road in the first image is 70%, and the confidence level of the secondary road in the first image is 20%.
[0069] In some embodiments, the second deep learning model includes a third deep learning sub-model and a fourth deep learning sub-model;
[0070] The step of inputting the first recognition result and the first image into a second deep learning model to obtain a second recognition result includes:
[0071] Inputting the first recognition result and the first image into the third deep learning sub-model to obtain a first recognition sub-result, wherein the first recognition sub-result includes the confidence of roads belonging to a first category in the first image;
[0072] The first recognition result and the first image are input into the fourth deep learning sub-model to obtain a second recognition sub-result, wherein the second recognition sub-result includes the confidence of roads in the first image belonging to the second category, and the second recognition result includes the first recognition sub-result and the second recognition sub-result.
[0073] In an embodiment of the present application, since different categories of roads may be related, for example, the main road and the auxiliary road are a road setting method. At this time, it can be considered that the main road and the auxiliary road exist at the same time. If there is only one road, there is no need to call the road a main road or an auxiliary road. Therefore, the third deep learning sub-model and the fourth deep learning sub-model can be used to detect the conditions of different categories of roads respectively, so that the categories of each road can be comprehensively judged in combination with the detection conditions of different categories of roads to improve the accuracy of the judgment. Among them, the first category and the second category can be determined according to the actual scene requirements. Exemplarily, the first category can be a main road and the second category can be an auxiliary road, or the first category can be an ordinary lane and the second category can be a bus-only lane, and so on.
[0074] For example, in one example, the confidence of the main road in the first image in the first recognition sub-result is 70%, and the confidence of the auxiliary road in the first image in the second recognition sub-result is 20%. At this time, since the confidence of the auxiliary road is low, it can be considered that there is no auxiliary road in the first image, and there may be only one road.
[0075] If the confidence of the main road in the first image in the first recognition sub-result is 10%, and the confidence of the auxiliary road in the first image in the second recognition sub-result is 80%, since the confidence of the main road is low at this time, it can be considered that there is no main road in the first image, and there may be only one road.
[0076] If the confidence level of the main road in the first image in the first recognition sub-result is 70%, and the confidence level of the auxiliary road in the first image in the second recognition sub-result is 80%, it can be considered that the main road and the auxiliary road exist in the first image.
[0077] Step S103: determining the road category in the first image according to the second recognition result.
[0078] After the second recognition result is obtained, the road category in the first image may be determined according to the confidence of the road category in the second recognition result.
[0079] For example, the road category in the first image can be determined based on the recognition of different road categories in the second recognition result through structures such as the fully connected layer in the second deep learning model. The road category in the first image can also be determined based on the second recognition result through other preset rules.
[0080] In some embodiments, the road categories include main roads and secondary roads, and the sign recognition results include recognition results of road lines and recognition results of non-road features between the main roads and the secondary roads.
[0081] In actual driving, drivers often have to choose a main road or a secondary road at a fork in the road, and the navigation system also needs to determine the navigation route based on whether the vehicle is currently on the main road or the secondary road. Therefore, in actual driving, it is very necessary to accurately identify whether the current vehicle is on the main road or the secondary road.
[0082] However, since the navigation system may have problems such as skipping points and insufficient accuracy, the navigation system is often unable to accurately determine whether the current vehicle is on a main road or a secondary road, and usually requires the driver to manually confirm whether the vehicle is on a main road or a secondary road.
[0083] In the embodiment of the present application, the first deep learning model can be used to identify roads in the first image, and to identify road lines in the first image and non-road features between the main road and the secondary road. Therefore, based on the road lines and the road recognition results, the second deep learning model can be used to more accurately identify the distribution of roads, and combined with the non-road features between the main road and the secondary road, it can be determined whether there is an isolation area between the roads, thereby more accurately judging whether there are main roads and secondary roads in the first image.
[0084] In an embodiment of the present application, a first recognition result for the first image can be obtained through a first deep learning model, so that the road conditions and signs in the first image can be identified, so that the first deep learning model can be used to achieve a preliminary perception of the road and related environment in the first image. Then, the first recognition result and the first image are input into a second deep learning model to obtain a second recognition result, and the second recognition result includes the confidence of the category of the road identified in the first image; at this time, the category of the road in the first image can be more accurately identified by the second deep learning model in combination with the preliminary perception of the road and related environment in the first image by the first deep learning model, and a second recognition result can be obtained, so that the category of the road in the first image can be more accurately determined based on the second recognition result.
[0085] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.
[0086] Corresponding to the method for determining the road category in the above embodiment, Figure 4 A structural block diagram of a device for determining a road category provided in an embodiment of the present application is shown. For ease of explanation, only the portion related to the embodiment of the present application is shown.
[0087] Reference Figure 4 , the device 4 for determining the road category comprises:
[0088] A first recognition module 401 is used to input a first image into a first deep learning model to obtain a first recognition result for the first image, where the first recognition result includes a road recognition result and a sign recognition result in the first image;
[0089] A second recognition module 402, configured to input the first recognition result and the first image into a second deep learning model to obtain a second recognition result, wherein the second recognition result includes a confidence level of a category of a road recognized in the first image;
[0090] The determination module 403 is used to determine the road category in the first image according to the second recognition result.
[0091] Optionally, the road categories include main roads and secondary roads, and the sign recognition results include recognition results of road lines and recognition results of non-road features between the main roads and the secondary roads.
[0092] Optionally, the first deep learning model includes a first deep learning sub-model and a second deep learning sub-model;
[0093] The first identification module 401 includes:
[0094] A first recognition unit, configured to input the first image into the first deep learning sub-model to obtain a road surface recognition result in the first image;
[0095] The second recognition unit is used to input the road surface recognition result and the first image into the second deep learning sub-model to obtain a first recognition result for the first image.
[0096] Optionally, the second deep learning model includes a third deep learning sub-model and a fourth deep learning sub-model;
[0097] The second identification module 402 includes:
[0098] Inputting the first recognition result and the first image into the third deep learning sub-model to obtain a first recognition sub-result, wherein the first recognition sub-result includes the confidence of roads belonging to a first category in the first image;
[0099] The first recognition result and the first image are input into the fourth deep learning sub-model to obtain a second recognition sub-result, wherein the second recognition sub-result includes the confidence of roads in the first image belonging to the second category, and the second recognition result includes the first recognition sub-result and the second recognition sub-result.
[0100] Optionally, the device 4 further comprises:
[0101] An acquisition module is used to acquire a target video captured by a camera on a specified vehicle;
[0102] The determination module is used to determine the first image from the target video, where the first image contains the front driving scene information of the designated vehicle.
[0103] Optionally, the identification result includes a road line identification result and an auxiliary identification result, the road line identification result includes the road line identified by the road line identification code, the auxiliary identification result includes the auxiliary identification obtained by the auxiliary identification code, and in the first identification result, the road line identification code and the auxiliary identification code are arranged in a specified order, and the specified order is determined according to the position information of the road line and the auxiliary identification in the first image respectively.
[0104] In an embodiment of the present application, a first recognition result for the first image can be obtained through a first deep learning model, so that the road conditions and signs in the first image can be identified, so that the first deep learning model can be used to achieve a preliminary perception of the road and related environment in the first image. Then, the first recognition result and the first image are input into a second deep learning model to obtain a second recognition result, and the second recognition result includes the confidence of the category of the road identified in the first image; at this time, the category of the road in the first image can be more accurately identified by the second deep learning model in combination with the preliminary perception of the road and related environment in the first image by the first deep learning model, and a second recognition result can be obtained, so that the category of the road in the first image can be more accurately determined based on the second recognition result.
[0105] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.
[0106] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the above-mentioned device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.
[0107] Figure 5 This is a schematic diagram of the structure of a terminal device provided in an embodiment of the present application. Figure 5 As shown, the terminal device 5 of this embodiment includes: at least one processor 50 ( Figure 5 Only one is shown), a memory 51, and a computer program 52 stored in the memory 51 and executable on at least one processor 50. When the processor 50 executes the computer program 52, the steps in any of the above-mentioned method embodiments for determining the road category are implemented.
[0108] The terminal device 5 may be a server, a mobile phone, a wearable device, an augmented reality (AR) / virtual reality (VR) device, a desktop computer, a notebook, a desktop computer, a handheld computer, or other computing devices. The terminal device may include, but is not limited to, a processor 50 and a memory 51. Those skilled in the art will appreciate that Figure 5 It is only an example of the terminal device 5 and does not constitute a limitation on the terminal device 5. It may include more or fewer components than shown in the figure, or combine certain components, or different components, for example, it may also include input devices, output devices, network access devices, etc. Among them, the above-mentioned input devices may include a keyboard, a touchpad, a fingerprint acquisition sensor (for collecting the user's fingerprint information and fingerprint direction information), a microphone, a camera, etc., and the output device may include a display, a speaker, etc.
[0109] The processor 50 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.
[0110] In some embodiments, the memory 51 may be an internal storage unit of the terminal device 5, such as a hard disk or memory of the terminal device 5. In other embodiments, the memory 51 may also be an external storage device of the terminal device 5, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the terminal device 5. Further, the memory 51 may also include both an internal storage unit and an external storage device of the terminal device 5. The memory 51 is used to store an operating system, an application program, a boot loader, data, and other programs, such as the program code of the computer program. The memory 51 may also be used to temporarily store data that has been output or is to be output.
[0111] In addition, although not shown, the terminal device 5 may also include a network connection module, such as a Bluetooth module, a Wi-Fi module, a cellular network module, etc., which will not be described in detail here.
[0112] In the embodiment of the present application, when the processor 50 executes the computer program 52 to implement the steps in any of the above-mentioned methods for determining the road category, in the embodiment of the present application, the first recognition result for the first image can be obtained through the first deep learning model, so that the road conditions and signs in the first image can be identified, so that the first deep learning model can be used to achieve a preliminary perception of the road and related environment in the first image. Then, the first recognition result and the first image are input into the second deep learning model to obtain a second recognition result, and the second recognition result includes the confidence of the category of the road identified in the first image; at this time, the category of the road in the first image can be more accurately identified by the second deep learning model in combination with the preliminary perception of the road and related environment in the first image by the first deep learning model, and a second recognition result can be obtained, so that the road category in the first image can be more accurately determined based on the second recognition result.
[0113] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.
[0114] An embodiment of the present application provides a computer program product. When the computer program product runs on a terminal device, the terminal device can implement the steps in the above-mentioned method embodiments when executing the computer program product.
[0115] If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program. The above-mentioned computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the above-mentioned computer program includes computer program code, and the above-mentioned computer program code can be in source code form, object code form, executable file or some intermediate form. The above-mentioned computer-readable medium may at least include: any entity or device that can carry the computer program code to the camera / terminal device, recording medium, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, RandomAccess Memory), electric carrier signal, telecommunication signal and software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.
[0116] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.
[0117] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.
[0118] In the embodiments provided in the present application, it should be understood that the disclosed devices / network equipment and methods can be implemented in other ways. For example, the device / network equipment embodiments described above are merely schematic. For example, the division of the above modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.
[0119] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0120] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some of the technical features therein by equivalents. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.
Claims
1. A method for determining a road category, characterized in that: include: Inputting the first image into a first deep learning model to obtain a first recognition result for the first image, wherein the first recognition result includes a road recognition result and a sign recognition result in the first image; Inputting the first recognition result and the first image into a second deep learning model to obtain a second recognition result, wherein the second recognition result includes a confidence level of a category of a road recognized in the first image; determining a road category in the first image according to the second recognition result; The identification recognition result includes a road line recognition result and an auxiliary identification recognition result. The road line recognition result includes a road line identified by a road line identification code, and the auxiliary identification result includes an auxiliary identification obtained by an auxiliary identification code. In addition, in the first recognition result, the road line identification code and the auxiliary identification code are arranged in a specified order, and the specified order is determined according to the position information of the road line and the auxiliary identification in the first image respectively.
2. The method according to claim 1, characterized in that The road categories include main roads and secondary roads, and the sign recognition results include recognition results of road lines and recognition results of non-road features between the main roads and the secondary roads.
3. The method according to claim 1, characterized in that The first deep learning model includes a first deep learning sub-model and a second deep learning sub-model; The step of inputting the first image into a first deep learning model to obtain a first recognition result for the first image includes: Inputting the first image into the first deep learning sub-model to obtain a road surface recognition result in the first image; The road surface recognition result and the first image are input into the second deep learning sub-model to obtain a first recognition result for the first image.
4. The method according to claim 1, characterized in that The second deep learning model includes a third deep learning sub-model and a fourth deep learning sub-model; The step of inputting the first recognition result and the first image into a second deep learning model to obtain a second recognition result includes: Inputting the first recognition result and the first image into the third deep learning sub-model to obtain a first recognition sub-result, wherein the first recognition sub-result includes the confidence of roads belonging to a first category in the first image; The first recognition result and the first image are input into the fourth deep learning sub-model to obtain a second recognition sub-result, wherein the second recognition sub-result includes the confidence of roads in the first image belonging to the second category, and the second recognition result includes the first recognition sub-result and the second recognition sub-result.
5. The method according to claim 1, characterized in that Before inputting the first image into the first deep learning model to obtain a first recognition result for the first image, the method further includes: Get the target video captured by the camera on the specified vehicle; The first image is determined from the target video, wherein the first image contains the front driving scene information of the designated vehicle.
6. A device for determining a road category, characterized in that: include: A first recognition module, configured to input a first image into a first deep learning model to obtain a first recognition result for the first image, wherein the first recognition result includes a road recognition result and a sign recognition result in the first image; A second recognition module, configured to input the first recognition result and the first image into a second deep learning model to obtain a second recognition result, wherein the second recognition result includes a confidence level of a category of a road recognized in the first image; a determination module, configured to determine a road category in the first image according to the second recognition result; Among them, the identification recognition result includes a road line recognition result and an auxiliary identification recognition result. The road line recognition result includes a road line identified by a road line identification code, and the auxiliary identification result includes an auxiliary identification obtained by an auxiliary identification code. In addition, in the first recognition result, the road line identification code and the auxiliary identification code are arranged in a specified order, and the specified order is determined according to the position information of the road line and the auxiliary identification in the first image respectively.
7. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method for determining the road category according to any one of claims 1 to 5 is implemented.
8. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method for determining the road category according to any one of claims 1 to 5 is implemented.
