A positioning method, device and electronic device based on scene semantic analysis

Abstract

The present invention discloses a positioning method, device and electronic device based on scene semantic analysis, wherein the method comprises: obtaining a scene image of a vehicle in a process of driving according to the positioning result of a current positioning satellite; inputting the scene image into a target convolutional neural network for training and parsing the scene image through the target convolutional neural network to obtain category information of an identifier contained in the scene image; determining a reference positioning satellite according to the category information of the identifier, the reference positioning satellite representing other satellites that can identify the identifier except the current positioning satellite; determining the positioning accuracy of the current positioning satellite and the reference positioning satellite for the identifier contained in the scene information; selecting a satellite whose positioning accuracy meets a target condition from the current positioning satellite and the reference positioning satellite to respond to the positioning operation during the vehicle driving process.

Description

Technical Field

[0001] The present invention relates to the field of positioning technology, and in particular to a positioning method, device and electronic equipment based on scene semantic analysis. Background Art

[0002] At present, the positioning accuracy of autonomous vehicles depends on high-precision positioning methods. The current mainstream multi-sensor positioning methods are mainly divided into two categories: one is the positioning and mapping (SLAM) technology based on visual information, and the other is the SLAM positioning technology based on laser radar. Due to the characteristics of laser radar itself, SLAM positioning technology based on laser radar is less dependent on the environment, and can still maintain high positioning accuracy in foggy or rainy and snowy weather, and also performs well under conditions where the GPS signal is weak. However, compared with other positioning methods, while laser radar has higher accuracy, it has a high cost and is easily contaminated by multiple vehicle signals, so the mainstream method for autonomous vehicles is still visual SLAM positioning technology.

[0003] Visual SLAM positioning technology combines the traditional visual processing method with the SLAM structure, inputs the scene image to determine the current vehicle's location information, but visual SLAM is easily affected by the weather and there will be misjudgments for similar buildings in different locations. At present, although the fixed GPS or GNSS positioning methods used by autonomous vehicles are relatively mature, the accuracy of satellite positioning will be affected by different geographical environments. For example, in urban canyons, there will be multipath effects, unstable positioning, and the positioning accuracy will also be greatly reduced near some highways, viaducts and tunnels. Therefore, it is urgent to propose a new positioning method to improve the satellite positioning accuracy of vehicles. Summary of the invention

[0004] Therefore, the present invention provides a positioning method, device and electronic device based on scene semantic analysis to improve the positioning accuracy of satellites for vehicles.

[0005] According to a first aspect, an embodiment of the present invention discloses a positioning method based on scene semantic analysis, including: obtaining a scene image of a vehicle during driving according to a positioning result of a current positioning satellite; inputting the scene image into a target convolutional neural network for training and parsing the scene image through the target convolutional neural network to obtain category information of a marker contained in the scene image; determining a reference positioning satellite based on the category information of the marker, the reference positioning satellite representing other satellites that can identify the marker except the current positioning satellite; determining the positioning accuracy of the current positioning satellite and the reference positioning satellite for the marker contained in the scene information; and selecting a satellite whose positioning accuracy meets the target condition from the current positioning satellite and the reference positioning satellite to respond to the positioning operation during vehicle driving.

[0006] Optionally, a satellite with a positioning accuracy that meets the target conditions is selected from the current positioning satellite and the reference positioning satellite to respond to the positioning operation during vehicle driving, including: when the positioning accuracy of the reference positioning satellite is greater than the positioning accuracy of the current positioning satellite, the reference positioning satellite is used to replace the current positioning satellite, so that the vehicle drives according to the positioning result of the reference positioning satellite; when the positioning accuracy of the reference positioning satellite is less than the positioning accuracy of the current positioning satellite, the vehicle is controlled to continue driving according to the positioning result of the current positioning satellite.

[0007] Optionally, the method further includes: determining whether the scene currently located by the vehicle exists in the digital map for navigating the vehicle according to the parsed scene information; if not, obtaining the road information and traffic sign information corresponding to the scene information from the editable digital map according to the current positioning result of the vehicle; fusing the scene information and the road information and traffic sign information corresponding to the scene information and adding them to the digital map;

[0008] Optionally, the method further comprises: jointly positioning the vehicle using a satellite whose positioning accuracy meets a target condition and the digital map.

[0009] Optionally, the target convolutional neural network includes: a convolutional layer, a pooling layer and a fully connected layer; wherein the convolution kernel size of the convolutional layer is 3*3, the number of convolution kernels is 32, the pooling window size in the pooling layer is 3*3, and the activation function of the fully connected layer is a ReLU function.

[0010] According to the second aspect, an embodiment of the present invention also discloses a positioning device based on scene semantic analysis, including: a first acquisition module, used to acquire a scene image of a vehicle during driving according to the positioning result of a current positioning satellite; a second acquisition module, used to input the scene image into a target convolutional neural network for training and parse the scene image through the target convolutional neural network to obtain category information of the marker contained in the scene image; a first determination module, used to determine a reference positioning satellite according to the category information of the marker, the reference positioning satellite representing other satellites that can identify the marker except the current positioning satellite; a second determination module, used to determine the positioning accuracy of the current positioning satellite and the reference positioning satellite for the marker contained in the scene information; a first positioning module, used to select a satellite whose positioning accuracy meets the target condition from the current positioning satellite and the reference positioning satellite to respond to the positioning operation during vehicle driving.

[0011] Optionally, the first positioning module includes: a first positioning submodule, used to replace the current positioning satellite with the reference positioning satellite when the positioning accuracy of the reference positioning satellite is greater than the positioning accuracy of the current positioning satellite, so that the vehicle travels according to the positioning result of the reference positioning satellite; and a second positioning submodule, used to control the vehicle to continue traveling according to the positioning result of the current positioning satellite when the positioning accuracy of the reference positioning satellite is less than the positioning accuracy of the current positioning satellite.

[0012] Optionally, the device also includes: a third determination module, used to determine whether the scene currently located by the vehicle exists in the digital map for navigation of the vehicle based on the parsed scene information; a third acquisition module, used to, if not, acquire road information and traffic sign information corresponding to the scene information from the editable digital map based on the positioning result of the current vehicle; and an adding module, used to merge the scene information and the road information and traffic sign information corresponding to the scene information and add them to the digital map.

[0013] According to the third aspect, an embodiment of the present invention also discloses an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor performs the steps of the positioning method based on scene semantic analysis as described in the first aspect or any optional embodiment of the first aspect.

[0014] According to the fourth aspect, an embodiment of the present invention further discloses a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the positioning method based on scene semantic analysis as described in the first aspect or any optional embodiment of the first aspect.

[0015] The technical solution of the present invention has the following advantages:

[0016] The positioning method / device based on scene semantic analysis provided by the present invention obtains a scene image during the driving process of a vehicle according to the positioning result of a current positioning satellite, inputs the scene image into a target convolutional neural network for training, and analyzes the scene image through the target convolutional neural network to obtain the category information of the marker contained in the scene image, determines the reference positioning satellites other than the current positioning satellite that can identify the marker according to the category information of the marker, determines the positioning accuracy of the current positioning satellite and the reference positioning satellite for the marker contained in the scene information, and selects a satellite whose positioning accuracy meets the target condition from the current positioning satellite and the reference positioning satellite to respond to the positioning operation during the driving process of the vehicle; identifies the scene image collected during the driving process of the vehicle to determine the reference positioning satellite that can be used for positioning according to the identified marker, and selects a satellite whose positioning accuracy meets the requirements from the positioning satellite currently used by the vehicle and the reference positioning satellite to respond to the positioning operation during the driving process of the vehicle. Compared with the vehicle using fixed satellites for vehicle positioning from beginning to end, the solution in the present application can select satellites that meet the positioning accuracy requirements (such as the highest positioning accuracy) in real time to position the vehicle, and the positioning method is more flexible, which ensures the accuracy of vehicle positioning. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 is a flowchart of a specific example of a positioning method based on scene semantic analysis in an embodiment of the present invention;

[0019] Figure 2 is a specific network structure diagram of the target convolutional neural network in an embodiment of the present invention;

[0020] Figure 3 is a principle block diagram of a specific example of a positioning device based on scene semantic analysis in an embodiment of the present invention;

[0021] Figure 4FIG. 4 is a specific example diagram of an electronic device in an embodiment of the present invention. DETAILED DESCRIPTION

[0022] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0023] In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc., indicating the orientation or positional relationship, are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", and "third" are used for descriptive purposes only, and cannot be understood as indicating or implying relative importance.

[0024] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, it can also be the internal connection of two components, it can be a wireless connection, or it can be a wired connection. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0025] In addition, the technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0026] The embodiment of the present invention discloses a positioning method based on scene semantic analysis, such as Figure 1 As shown, the method comprises the following steps:

[0027] Step 101, obtaining a scene image of a vehicle in the process of driving according to the positioning result of the current positioning satellite.

[0028] Exemplarily, the scene during the driving process of the vehicle can be photographed by the on-board image acquisition device, and the collected scene image can be transmitted to the on-board processor by the on-board image transmission device. The embodiment of the present application does not limit the method of collecting scene images. The positioning satellite used by the vehicle during the current driving process can be a satellite of a pre-specified default group, and the pre-specified satellite is used as the current positioning satellite to respond to the positioning operation for the driving vehicle. The embodiment of the present application does not limit the type of the current positioning satellite.

[0029] Step 102: input the scene image into a target convolutional neural network for training and parse the scene image through the target convolutional neural network to obtain category information of the marker contained in the scene image.

[0030] Exemplarily, the category information of the markers contained in the scene image can be directly identified by the target convolutional neural network after real-time training of the target convolutional neural network based on the collected scene image. Since the driving scene faced by the vehicle during driving is relatively complex (such as foggy days, rainy days and other environmental factors may appear in the scene), compared to using the existing original image data to pre-train the neural network model to identify the category information of the markers in the scene image, the embodiment of the present application uses the scene image of the vehicle during driving collected in real time in the actual driving scene to train the target convolutional neural network model and outputs the recognition result according to the real-time trained target convolutional neural network model, so that the result of the identified category information of the marker is more accurate.

[0031] In order to improve the recognition efficiency of the target convolutional neural network model, the scale of the network structure used to output the category information of the markers contained in the scene image cannot be too large. As an optional embodiment of the present invention, the target convolutional neural network includes: a convolutional layer, a pooling layer and a fully connected layer; wherein the convolution kernel size of the convolution layer is 3*3, the number of convolution kernels is 32, the pooling window size in the pooling layer is 3*3, and the activation function of the fully connected layer is a ReLU function.

[0032] The convolution layer samples different areas of the image through different neurons, and shares the same weight matrix to obtain specific features. Each convolution kernel can judge a type of feature, so increasing the number of convolution kernels can extract more types of features. In the embodiment of the present application, the size of the convolution kernel is selected as: 3*3, and the number of convolution kernels is 32; the pooling layer is the process of data simplification. Since the image after convolution will have messy eigenvalues, and the image features will overlap due to the overlap of the convolution range, it is necessary to use the pooling layer to process the eigenvalues ​​and screen out the appropriate eigenvalues. Common pooling methods include average pooling, maximum pooling, overlapping pooling, and spatial pyramid pooling. In the embodiment of the present application, considering the on-board computing power and accuracy issues, the maximum pooling method is used as the pooling layer; at the same time, in order to ensure the stability of the gradient, there will be no diffusion or explosion phenomenon. In the target convolution neural network in the embodiment of the present application, ReLU is selected as the activation function to ensure that the variance can be reduced when it is too large in the activation stage. In addition, after each layer of convolution is completed, a standard function and a dropout layer are added. The norm function normalizes the input image information, with the mean close to 1 and the variance close to 0. The dropout layer can invalidate the input neurons with a given probability, reduce the number of intermediate features, and prevent the network from overfitting. The specific structure is as follows: Figure 2 The embodiment of the present application does not limit the structure of the target convolutional neural network model, and those skilled in the art may select a model with other structures according to the requirements for the output timeliness of the recognition results.

[0033] In order to improve the stability of the target convolutional neural network recognition results obtained by real-time training, in addition to relying on the acquired scene information, GPS information and scene digital map information can be further combined as auxiliary recognition verification to ensure the stability of the network. The target convolutional neural network structure used in the embodiment of the present application is small in scale, suitable for vehicle-mounted network training, can output recognition results in real time, and the training data set is small, which further reduces the demand for the use of vehicle-mounted computing power.

[0034] Step 103: Determine a reference positioning satellite according to the category information of the marker, where the reference positioning satellite represents other satellites that can identify the marker except the current positioning satellite.

[0035] Exemplarily, the method for determining the reference positioning satellite can be to receive the positioning information of the marker from different satellites based on the identified marker and its category information in the scene image through the base station, and the satellite or satellite combination that locates the marker constitutes a reference positioning satellite.

[0036] For example, after obtaining the category information of the markers in the scene image, a fast food restaurant "A" is identified, and all satellites that can monitor the fast food restaurant "A" can be recorded. The accuracy of the satellite positioning signal can then be calculated through different permutations and combinations to select the combined satellite as a reference positioning satellite; or any satellite that can locate the fast food restaurant "A" can be used as a reference positioning satellite.

[0037] The satellite combination method may be based on the number of positioning satellites or the type of satellites. For example, if there are 4 satellites that can locate a certain marker in the current scene, the reference positioning satellites can be obtained by arranging and combining the 4 satellites according to the number or category. The selection of the default group of satellites as the current positioning satellite may be the simplest positioning method, such as directly selecting one of the satellites that can locate a certain marker in the current scene as the default group.

[0038] When multiple markers are identified, they can also be further used to screen satellites. For example, when the same marker appears in two scenes that are close to each other, such as two fast food restaurants "A" that are very close to each other in a certain place, when the satellite fails to achieve accurate positioning and distinction, the satellites can be screened by comparing the scene elements around the same markers. For example, if satellite 1 detects that there is a hospital around fast food restaurant "A", and satellite 2 detects that there is a school around fast food restaurant "A", and hospital-related category information is also identified in the scene image, satellite 1 can be preferentially selected as the reference positioning satellite.

[0039] Step 104: Determine the positioning accuracy of the current positioning satellite and the reference positioning satellite for the marker included in the scene information. In the embodiment of the present application, the positioning accuracy is calculated by the following method:

[0040] Specifically, the three-dimensional position (x, y, z) of the scene marker and the clock difference δ between the receiver and GPS are obtained according to the pseudorange ρ t The basic positioning equation of four unknown variables, where the positioning equation is as follows:

[0041]

[0042] Among them, (x s ,y s ,z s ) is the satellite's position coordinate; c is the speed of light; δ t is the clock difference between the receiver and the GPS signal; ρ is the pseudorange between the receiver and the satellite obtained from the atmosphere and the clock difference; δ t,sis the clock difference between the satellite and GPS; I is the delay caused by the ionization reaction of the atmosphere; T is the delay caused by the convection phenomenon of the atmosphere, which can be estimated and determined according to the Saastamoinen model; ∈ is the delay caused by other unknown factors including noise, and other unknown factors can be regarded as Gaussian distribution and fitted to obtain ∈.

[0043] The pseudorange measurement is usually calculated by the receiver according to the following formula:

[0044] ρ=c(t u -t s ) (2)

[0045] Among them, t u The time displayed by the receiver, where the time of the receiver is directly displayed on the receiver in the form of code text and can be directly read from the receiver, t s is the time when the satellite sends the signal, t s Calculated according to the following formula:

[0046]

[0047] Among them, TOW is the GPS time displayed by the satellite navigation message; w is the number of words that the receiver can fully record in the current subframe; b is the number of complete bits that the receiver can distinguish in the current complete word; n is the number of complete C / A codes that the receiver can distinguish in the current complete bit; CP is the number of chips that the receiver has received in the current complete C / A code. All the above quantities can be calculated based on the satellite clock and are known variables.

[0048] Use Newton's iteration method to solve the corresponding equation, as shown below:

[0049]

[0050] And find the Jacobian matrix G. Suppose that in the kth iteration, the unit observation vector from the receiver to the satellite can be expressed as e s，k =[e s,k,x e s,k,y e s,k,z ] T , then G can be expressed as:

[0051]

[0052] After linearizing the equation, the least squares method can be used to solve the system of equations. This system of equations is represented by the Jacobian matrix and constants. The specific formula is shown below:

[0053]

[0054] Based on the equation, the iterative relationship between the previous step and the next step is obtained, as shown in the following formula:

[0055]

[0056] In the above formula (7), X k , Y k , Z k The test results of the scene marker positions at different times are as follows. When the position parameters obtained during the iterative process meet the accuracy requirements, the iteration can be stopped.

[0057] In the process of solving the equation, the positioning error vector can be obtained, which is specifically expressed as follows:

[0058]

[0059] Among them, ∈ x ,∈ y ,∈ z , The error vector ∈ ρ The errors caused by the positioning and timing.

[0060] Assuming that the satellite measurement errors are normally distributed, the covariance matrix of the positioning error can be obtained:

[0061]

[0062] Where H = (G T G) -1 It can be called a weight matrix, which is also a symmetric matrix. It represents the variance of the ranging error, and its mathematical meaning is ∈ ρ The variance value of (pseudorange measurement error).

[0063] The positioning accuracy can be directly determined by using the weight matrix in the covariance matrix of the positioning error. The positioning accuracy of the markers included in the scene identified by the current positioning satellite and the reference positioning satellite is calculated in the above manner.

[0064] Step 105: Select a satellite whose positioning accuracy meets the target condition from the current positioning satellite and the reference positioning satellite to respond to the positioning operation during the vehicle driving process.

[0065] Exemplarily, after determining the positioning accuracy of the current positioning satellite and the reference positioning satellite, the precision satellite can be selected to respond to the positioning operation according to the accuracy; or if the accuracy of the current positioning satellite and the reference positioning satellite both meet the accuracy requirements, the current positioning satellite can be used to position the vehicle and the reference positioning satellite can be used to simultaneously respond to the positioning operation on the vehicle.

[0066] The positioning method based on scene semantic analysis provided by the present invention obtains a scene image during the driving process of a vehicle according to the positioning result of a current positioning satellite, inputs the scene image into a target convolutional neural network for training, and analyzes the scene image through the target convolutional neural network to obtain the category information of the marker contained in the scene image, determines the reference positioning satellites other than the current positioning satellite that can identify the marker according to the category information of the marker, determines the positioning accuracy of the current positioning satellite and the reference positioning satellite for the marker contained in the scene information, and selects a satellite whose positioning accuracy meets the target condition from the current positioning satellite and the reference positioning satellite to respond to the positioning operation during the driving process of the vehicle; identifies the scene image collected during the driving process of the vehicle to determine the reference positioning satellite that can be used for positioning according to the identified marker, and selects a satellite whose positioning accuracy meets the requirements from the positioning satellite currently used by the vehicle and the reference positioning satellite to respond to the positioning operation during the driving process of the vehicle. Compared with the vehicle using fixed satellites for vehicle positioning from beginning to end, the solution in the present application can select satellites that meet the positioning accuracy requirements (such as the highest positioning accuracy) in real time to position the vehicle, and the positioning method is more flexible, which ensures the accuracy of vehicle positioning.

[0067] As an optional implementation of the present invention, step 105 includes: when the positioning accuracy of the reference positioning satellite is greater than the positioning accuracy of the current positioning satellite, the current positioning satellite is replaced by the reference positioning satellite, so that the vehicle travels according to the positioning result of the reference positioning satellite; when the positioning accuracy of the reference positioning satellite is less than the positioning accuracy of the current positioning satellite, the vehicle is controlled to continue to travel according to the positioning result of the current positioning satellite. By flexibly using positioning satellites with positioning accuracy, the accuracy of vehicle positioning is ensured, and the timeliness of positioning is improved compared to using multiple or multiple groups of satellites for positioning.

[0068] As an optional embodiment of the present invention, the method also includes: determining whether the scene currently located by the vehicle exists in the digital map for navigation of the vehicle based on the parsed scene information; if not, obtaining the road information and traffic sign information corresponding to the scene information from the editable digital map based on the positioning result of the current vehicle; and fusing the scene information and the road information and traffic sign information corresponding to the scene information and adding them to the digital map.

[0069] Exemplarily, the editable digital map may be OpenstreetMap. Since OpenstreetMap allows browsing, modification and use of geographic data, OpenstreetMap can be used to expand the digital map used during vehicle driving. The road information obtained from the editable digital map may include: the width of the lane line, the type of lane separator line, the type of lane arrow marking and the lane axis. The lane axis is located in the center of the lane and can play a role in lane positioning. The embodiment of the present application does not limit the types of road information and traffic sign information obtained, and those skilled in the art can determine it according to actual needs. The obtained scene information, traffic sign information, and road information are integrated into the digital map used during vehicle driving through a filtering algorithm to achieve the expansion of the digital map and ensure the accuracy and stability of subsequent vehicle positioning.

[0070] Compared with the traditional paper standard map, the extended digital map can use additional scene features, such as the topology, geometry and geographic location of the road to improve the stability and accuracy of positioning navigation. The digital map can also generate a file with a structure similar to the road network description file for subsequent positioning.

[0071] As an optional implementation manner of the present invention, the method further comprises: jointly positioning the vehicle using a satellite whose positioning accuracy meets the target condition and the digital map.

[0072] For example, after the digital map is expanded and improved, the expanded digital map and satellites whose positioning accuracy meets the target conditions can be combined to assist and supplement subsequent satellite positioning. For example, GPS can only recognize the existence of hospitals and fast food restaurants in the current scene, but some smaller components such as traffic cannot be recognized by GPS. At this time, the expanded digital map can be combined to obtain more detailed prior scene information, and the obtained GPS positioning results and digital map information can be used to assist and supplement the output results of the target neural network model to ensure the accuracy and stability of the output results of the target neural network model, and ensure the accuracy of the vehicle positioning results through the joint positioning of multiple information.

[0073] The embodiment of the present invention also discloses a positioning device based on scene semantic analysis, such as Figure 3 As shown, the device comprises:

[0074] The first acquisition module 301 is used to acquire a scene image when the vehicle is driving according to the positioning result of the current positioning satellite;

[0075] A second acquisition module 302 is used to input the scene image into a target convolutional neural network for training and parse the scene image through the target convolutional neural network to obtain category information of the marker contained in the scene image;

[0076] A first determination module 303 is used to determine a reference positioning satellite according to the category information of the marker, where the reference positioning satellite represents other satellites that can identify the marker except the current positioning satellite;

[0077] A second determination module 304 is used to determine the positioning accuracy of the current positioning satellite and the reference positioning satellite for the marker included in the scene information;

[0078] The first positioning module 305 is used to select a satellite whose positioning accuracy meets the target condition from the current positioning satellite and the reference positioning satellite to respond to the positioning operation during the vehicle driving process.

[0079] The positioning device based on scene semantic analysis provided by the present invention obtains a scene image during the driving process of a vehicle according to the positioning result of a current positioning satellite, inputs the scene image into a target convolutional neural network for training, and analyzes the scene image through the target convolutional neural network to obtain the category information of the marker contained in the scene image, determines the reference positioning satellites other than the current positioning satellite that can identify the marker according to the category information of the marker, determines the positioning accuracy of the current positioning satellite and the reference positioning satellite for the marker contained in the scene information, and selects a satellite whose positioning accuracy meets the target condition from the current positioning satellite and the reference positioning satellite to respond to the positioning operation during the driving process of the vehicle; identifies the scene image collected during the driving process of the vehicle to determine the reference positioning satellite that can be used for positioning according to the identified marker, and selects a satellite whose positioning accuracy meets the requirement from the positioning satellite currently used by the vehicle and the reference positioning satellite to respond to the positioning operation during the driving process of the vehicle. Compared with the vehicle using a fixed satellite for vehicle positioning from beginning to end, the solution in the present application can select a satellite that meets the positioning accuracy requirement (such as the highest positioning accuracy) in real time to position the vehicle, and the positioning method is more flexible, thereby ensuring the accuracy of vehicle positioning.

[0080] As an optional embodiment of the present invention, the first positioning module 305 includes: a first positioning submodule, which is used to replace the current positioning satellite with the reference positioning satellite when the positioning accuracy of the reference positioning satellite is greater than the positioning accuracy of the current positioning satellite, so that the vehicle travels according to the positioning result of the reference positioning satellite; and a second positioning submodule, which is used to control the vehicle to continue traveling according to the positioning result of the current positioning satellite when the positioning accuracy of the reference positioning satellite is less than the positioning accuracy of the current positioning satellite.

[0081] As an optional embodiment of the present invention, the device also includes: a third determination module, used to determine whether the scene currently located by the vehicle exists in the digital map for navigation of the vehicle based on the parsed scene information; a third acquisition module, used to determine whether the scene currently located by the vehicle exists in the digital map for navigation of the vehicle based on the parsed scene information; an adding module, used to merge the scene information and the road information and traffic sign information corresponding to the scene information and add them to the digital map.

[0082] As an optional implementation of the present invention, the device further includes a second positioning module, which is used to jointly position the vehicle using satellites whose positioning accuracy meets the target condition and the digital map.

[0083] As an optional embodiment of the present invention, the target convolutional neural network includes: a convolutional layer, a pooling layer and a fully connected layer; wherein the convolution kernel size of the convolutional layer is 3*3, the number of convolution kernels is 32, the pooling window size in the pooling layer is 3*3, and the activation function of the fully connected layer is a ReLU function.

[0084] The embodiment of the present invention further provides an electronic device, such as Figure 4 As shown, the electronic device may include a processor 401 and a memory 402, wherein the processor 401 and the memory 402 may be connected via a bus or other means. Figure 4 The example of connecting through bus is taken in the following.

[0085] The processor 401 may be a central processing unit (CPU). The processor 401 may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or a combination of the above chips.

[0086] The memory 402, as a non-transient computer-readable storage medium, can be used to store non-transient software programs, non-transient computer executable programs and modules, such as program instructions / modules corresponding to the positioning method based on scene semantic parsing in the embodiment of the present invention. The processor 401 executes various functional applications and data processing of the processor by running the non-transient software programs, instructions and modules stored in the memory 402, that is, realizing the positioning method based on scene semantic parsing in the above method embodiment.

[0087] The memory 402 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required by at least one function; the data storage area may store data created by the processor 401, etc. In addition, the memory 402 may include a high-speed random access memory, and may also include a non-transitory memory, such as at least one disk storage device, a flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory 402 may optionally include a memory remotely arranged relative to the processor 401, and these remote memories may be connected to the processor 401 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0088] The one or more modules are stored in the memory 402, and when executed by the processor 401, the following is performed: Figure 1 The positioning method based on scene semantic analysis in the illustrated embodiment.

[0089] For details of the above electronic equipment, please refer to Figure 1 The corresponding related descriptions and effects in the illustrated embodiments can be understood and will not be repeated here.

[0090] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the storage medium can be a disk, an optical disk, a read-only memory (ROM), a random access memory (RAM), a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD), etc.; the storage medium can also include a combination of the above-mentioned types of memory.

[0091] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A positioning method based on scene semantic analysis, characterized in that: include: Obtaining scene images of the vehicle while it is traveling according to the positioning results of the current positioning satellite; Inputting the scene image into a target convolutional neural network for training and parsing the scene image through the target convolutional neural network to obtain category information of the marker contained in the scene image; Determine a reference positioning satellite according to the category information of the marker, where the reference positioning satellite represents other satellites that can identify the marker except the current positioning satellite; Determining the positioning accuracy of the current positioning satellite and the reference positioning satellite for the marker included in the scene information; Selecting a satellite whose positioning accuracy meets the target condition from the current positioning satellite and the reference positioning satellite to respond to the positioning operation during the vehicle driving; The positioning operation of selecting a satellite whose positioning accuracy meets the target condition from the current positioning satellite and the reference positioning satellite to respond to the vehicle during driving includes: When the positioning accuracy of the reference positioning satellite is greater than the positioning accuracy of the current positioning satellite, the current positioning satellite is replaced by the reference positioning satellite, so that the vehicle travels according to the positioning result of the reference positioning satellite; When the positioning accuracy of the reference positioning satellite is less than the positioning accuracy of the current positioning satellite, the vehicle is controlled to continue to travel according to the positioning result of the current positioning satellite; The method further comprises: Determine, based on the parsed scene information, whether the scene the vehicle is currently in exists in the digital map used for navigation of the vehicle; If not, obtaining road information and traffic sign information corresponding to the scene information from an editable digital map according to the current vehicle positioning result; The scene information and the road information and traffic sign information corresponding to the scene information are integrated and added to the digital map; The method further comprises: The vehicle is jointly positioned using a satellite whose positioning accuracy meets the target condition and the digital map.

2. The method according to claim 1, characterized in that The target convolutional neural network includes: a convolutional layer, a pooling layer and a fully connected layer; wherein the convolution kernel size of the convolutional layer is 3*3, the number of convolution kernels is 32, the pooling window size in the pooling layer is 3*3, and the activation function of the fully connected layer is the ReLU function.

3. A positioning device based on scene semantic analysis, characterized in that: include: A first acquisition module is used to acquire a scene image when the vehicle is driving according to the positioning result of the current positioning satellite; A second acquisition module is used to input the scene image into a target convolutional neural network for training and parse the scene image through the target convolutional neural network to obtain category information of the marker contained in the scene image; A first determination module is used to determine a reference positioning satellite according to the category information of the marker, where the reference positioning satellite represents other satellites that can identify the marker except the current positioning satellite; A second determination module is used to determine the positioning accuracy of the current positioning satellite and the reference positioning satellite for the marker included in the scene information; A first positioning module is used to select a satellite whose positioning accuracy meets the target condition from the current positioning satellite and the reference positioning satellite to respond to the positioning operation during the vehicle driving process; The first positioning module includes: A first positioning submodule, configured to replace the current positioning satellite with the reference positioning satellite when the positioning accuracy of the reference positioning satellite is greater than the positioning accuracy of the current positioning satellite, so that the vehicle travels according to the positioning result of the reference positioning satellite; A second positioning submodule, configured to control the vehicle to continue driving according to the positioning result of the current positioning satellite when the positioning accuracy of the reference positioning satellite is less than the positioning accuracy of the current positioning satellite; A third determination module is used to determine whether the scene currently located by the vehicle exists in the digital map for navigating the vehicle according to the parsed scene information; A third acquisition module is used to, if not, acquire road information and traffic sign information corresponding to the scene information from an editable digital map according to the positioning result of the current vehicle; An adding module, used for fusing the scene information and the road information and traffic sign information corresponding to the scene information and adding the result to the digital map; The second positioning module is used to jointly position the vehicle using satellites whose positioning accuracy meets the target conditions and the digital map.

4. An electronic device, characterized in that: include: at least one processor; And a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor performs the steps of the positioning method based on scene semantic parsing as described in any one of claims 1-2.

5. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the positioning method based on scene semantic analysis as described in any one of claims 1 to 2 are implemented.

Images