Matching evaluation method for putting target agent to current investigation road section and related device

Abstract

The invention discloses a matching evaluation method for putting a target agent to a current investigation road section and a related device. The method comprises the following steps: acquiring road information of a current reconnaissance road section and key parameters of a target agent, determining road characterization parameters based on image information related to point cloud in the road information, and performing parameter matching evaluation on the road characterization parameters of the current reconnaissance road section and the key parameters of the target agent; on the basis of image information related to RGB in the road information, verifying the matching evaluation result; and according to the verification result, evaluating the adaptation degree and the delivery risk of the target agent in the current investigation road section. Therefore, manual evaluation specialty does not need to be relied on, automatic intelligent matching evaluation is realized, evaluation cost and time consumption are reduced, and evaluation accuracy is high.

Description

Technical Field

[0001] The present application relates to the field of computer technology, specifically to technical fields such as autonomous driving control and road survey and evaluation, and in particular to a matching evaluation method and related devices for placing a target intelligent body on a currently surveyed road section. Background Art

[0002] Currently, there are an increasing number of devices equipped with autonomous driving systems, particularly those used in non-municipal and non-public roads, such as parks, industrial parks, and scenic spots, which have become the primary operating environments for functional autonomous driving equipment. However, compared to municipal non-public roads, these non-public roads in these scenarios suffer from incomplete data, inconsistent design standards, and complex infrastructure.

[0003] Therefore, to assess whether the non-public roads within a scenario meet the requirements for autonomous driving equipment operation, a common approach to obtaining a relatively accurate assessment is to arrange for experienced engineers to conduct on-site surveys of the non-public roads where autonomous driving equipment will be deployed. This involves simply measuring basic data on the non-public roads, such as subsidence height, length, width, height, slope, obstacles, and obstructions. Engineers can also combine relevant basic information about the autonomous driving equipment and its autonomous driving capabilities, such as length, width, height, gradeability, turning radius, and autonomous driving system performance, to comprehensively evaluate the feasibility of using the autonomous driving equipment on non-public roads within the scenario.

[0004] However, the above manual evaluation scheme is heavily dependent on experienced engineers, has high evaluation costs and a long evaluation cycle, and the professionalism and accuracy of the evaluation cannot be guaranteed. Summary of the Invention

[0005] This application provides a matching evaluation method and related devices for placing a target intelligent body on the current survey road section to solve the problems of existing road survey and evaluation schemes relying on manual evaluation, which is unprofessional, costly, time-consuming, and has low evaluation accuracy.

[0006] The technical solution is as follows:

[0007] In the first aspect, a matching evaluation method for placing a target intelligent agent on a currently surveyed road section is provided, comprising:

[0008] Obtaining road information of the current surveyed road section and key parameters of the target intelligent body, wherein the key parameters include at least the size and equipment performance of the target intelligent body;

[0009] Determining road characterization parameters based on image information related to the point cloud in the road information, wherein the road characterization parameters include at least: the slope, length, width, height, and obstacles of the current surveyed road section;

[0010] Perform parameter matching evaluation on the road characterization parameters of the current surveyed road section and the key parameters of the target intelligent agent;

[0011] Verifying the matching evaluation result based on the RGB-related image information in the road information;

[0012] Based on the verification results, the suitability and deployment risk of the target intelligent agent for use in the current survey section are evaluated.

[0013] In one possible implementation, the road characterization parameters of the current surveyed road section are evaluated for parameter matching with the key parameters of the target intelligent agent, specifically including:

[0014] Constructing a digital twin model of the current surveyed road section and the target intelligent agent to map the current surveyed road section and the target intelligent agent into a virtual environment;

[0015] Based on the road characterization parameters of the current surveyed road section and the key parameters of the target intelligent agent, simulating a variety of different driving behaviors;

[0016] The matching evaluation of different parameters is carried out based on the simulation results.

[0017] In one possible implementation, verifying the matching evaluation result based on the RGB-related image information in the road information specifically includes:

[0018] For the parameter items that do not match or are ambiguous in the matching evaluation results, the RGB image of the corresponding road section is searched from the image database established based on RGB-related image information;

[0019] Analyze and verify the matching results based on the concrete content in the found RGB image.

[0020] In one possible implementation, obtaining the road information of the currently surveyed road section specifically includes:

[0021] In non-public road scenarios, obtain the point cloud image and RGB image of the current survey section and perform image preprocessing operations;

[0022] Based on the curb recognition model trained by the Faster-RCNN algorithm, the RGB curb of the current surveyed road section is identified from the RGB image after preprocessing operation;

[0023] Extracting a point cloud curb of the current surveyed road section from the point cloud image based on a road window corresponding to a set single-directional scanning line;

[0024] Using different confidence evaluation indexes respectively, performing confidence evaluation on the RGB curb and the point cloud curb;

[0025] If the confidence evaluation results of the RGB curb and the confidence evaluation results of the point cloud curb are both greater than the set first threshold, the survey is determined to be reliable, and the road information of the current surveyed section is determined based on the point cloud curb and the RGB curb;

[0026] If at least one of the confidence evaluation result of the RGB curb and the confidence evaluation result of the point cloud curb is not greater than the set first threshold, it is determined that the current survey is unreliable and the above survey is performed again.

[0027] In a possible implementation, based on a set road window corresponding to a single-directional scanning line, extracting a point cloud curb of the current surveyed road section from the point cloud image specifically includes:

[0028] Converting the point cloud image into a linear scan point cloud map comprising a series of ordered two-dimensional scan line data sets; wherein the two-dimensional scan lines are perpendicular to the survey travel direction;

[0029] Extracting a scan line cross-section corresponding to each two-dimensional scan line from the linear scan point cloud map as a road window;

[0030] Estimate the curb boundaries on the left and right sides based on the discrete point clouds on the left and right sides of each road window, respectively, wherein the curb boundaries include the curb boundary position and height;

[0031] The curb boundaries of each road window are summarized to obtain the point cloud curb of the current survey section.

[0032] In a possible implementation, determining the road information of the current surveyed road section based on the point cloud curb and the RGB curb specifically includes:

[0033] Cutting the RGB image based on the RGB curb, wherein the RGB image retained after cutting includes a curb image area and a road surface image area between the curbs;

[0034] Cutting the point cloud image based on the point cloud curb, wherein the point cloud image retained after cutting includes a curb image area and a road surface image area between the curbs;

[0035] The RGB image and the point cloud image retained after cutting are used as the road information of the current survey section.

[0036] In a possible implementation, cutting the point cloud image based on the point cloud curb specifically includes:

[0037] The image area outside the point cloud curb in the point cloud image is cut off and removed, and the image area within the point cloud image that is set to a second threshold away from the road surface is cut off and removed.

[0038] In a second aspect, a matching evaluation device for placing a target intelligent agent on a currently surveyed road section is provided, comprising:

[0039] An acquisition module is used to acquire road information of the current surveyed road section and key parameters of the target intelligent body, wherein the key parameters include at least the size and equipment performance of the target intelligent body;

[0040] a determination module, configured to determine road characterization parameters based on image information related to the point cloud in the road information, the road characterization parameters including at least: a slope, length, width, height, and obstacles of a currently surveyed road section;

[0041] A matching module, configured to perform parameter matching evaluation between the road characterization parameters of the current surveyed road section and the key parameters of the target intelligent agent;

[0042] a verification module, configured to verify the matching evaluation result based on the RGB-related image information in the road information;

[0043] The evaluation module is used to evaluate the suitability and deployment risk of the target intelligent agent for use in the current survey section based on the verification results.

[0044] According to a third aspect, an electronic device is provided, including:

[0045] at least one processor; and

[0046] a memory communicatively connected to the at least one processor; wherein,

[0047] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of any possible implementation manner and the aspects described above.

[0048] In a fourth aspect, a computer-readable storage medium is provided, wherein the storage medium stores at least one instruction, and the at least one instruction is loaded and executed by a processor to implement the method of the above-mentioned aspect and any possible implementation manner.

[0049] In a fifth aspect, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the above-mentioned aspects and any possible implementation method.

[0050] The beneficial effects of the technical solution provided by this application include at least:

[0051] As can be seen from the above technical solution, the embodiment of the present application obtains the road information of the current survey section and the key parameters of the target intelligent agent, determines the road characterization parameters based on the image information related to the point cloud in the road information, and performs parameter matching evaluation on the road characterization parameters of the current survey section and the key parameters of the target intelligent agent; verifies the matching evaluation results based on the RGB-related image information in the road information; and evaluates the suitability and deployment risk of the target intelligent agent for use in the current survey section based on the verification results. As a result, there is no need to rely on the professionalism of manual evaluation, and automatic intelligent matching evaluation is achieved, which reduces evaluation costs and time consumption, and has high evaluation accuracy.

[0052] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0054] Figure 1 This is a schematic diagram of the steps of a matching evaluation method for placing a target intelligent agent on a currently surveyed road section provided in an embodiment of the present application.

[0055] Figure 2 This is a schematic diagram of the steps for obtaining road information of the current survey section provided in an embodiment of the present application.

[0056] Figure 3 This is a simplified schematic diagram of the smart handheld device provided in an embodiment of the present application.

[0057] Figure 4a This is a linear scanning point cloud image provided by an embodiment of the present application.

[0058] Figure 4b This is a scan line cross-sectional view provided by an embodiment of the present application.

[0059] Figure 5a This is a schematic diagram of the survey process of a non-public road provided in an embodiment of the present application.

[0060] Figure 5b This is a schematic diagram of the survey and evaluation process of non-public roads provided in an embodiment of the present application.

[0061] Figure 6This is a structural block diagram of a matching evaluation device for placing a target intelligent agent on a currently surveyed road section, provided by an embodiment of the present application.

[0062] Figure 7 This is a structural diagram of a matching evaluation system for placing a target intelligent agent on a currently surveyed road section, provided by an embodiment of the present application.

[0063] Figure 8 This is a block diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0064] The following description of exemplary embodiments of the present application is provided in conjunction with the accompanying drawings, which include various details of the embodiments of the present application to facilitate understanding. These details should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present application. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0065] Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0066] It should be noted that the terminal devices involved in the embodiments of the present application may include but are not limited to mobile phones, personal digital assistants (PDAs), wireless handheld devices, tablet computers and other smart devices; display devices may include but are not limited to personal computers, televisions and other devices with display functions.

[0067] In this document, the term "and / or" simply describes a relationship between related objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A exists alone, A and B exist simultaneously, or B exists alone. Furthermore, the character " / " in this document generally indicates that the related objects are in an "or" relationship.

[0068] In view of the fact that manual survey solutions rely heavily on experienced engineers, have high survey costs and long survey cycles, and cannot guarantee survey professionalism and accuracy, this application proposes a survey solution for non-public roads. The main inventive concept is to obtain the road information of the current survey section and the key parameters of the target intelligent body, determine the road characterization parameters based on the image information related to the point cloud in the road information, and perform parameter matching evaluation on the road characterization parameters of the current survey section and the key parameters of the target intelligent body; verify the matching evaluation results based on the RGB-related image information in the road information; and evaluate the adaptability and deployment risk of the target intelligent body in the current survey section based on the verification results. Therefore, there is no need to rely on manual evaluation professionalism, and automatic intelligent matching evaluation is achieved, which reduces evaluation costs and time consumption, and has high evaluation accuracy.

[0069] Reference Figure 1 The figure shows a schematic diagram of the steps of a matching evaluation method for placing a target intelligent agent on the current survey section provided by an embodiment of the present application. It should be understood that the execution subject of the matching evaluation method can be a matching evaluation device, which can be a software module or hardware device with computer data processing, storage and other functions as well as data acquisition functions, such as a computer, tablet computer, smartphone or smart wearable device with an acquisition module. Specifically, the survey device can be a backend server, which can be a cloud-based backend, a distributed backend, or an ordinary server, and this application is not limited to this.

[0070] like Figure 1 As shown, the matching evaluation method for placing the target agent on the current survey section may include the following steps:

[0071] Step 102: Obtain road information of the current surveyed road section and key parameters of the target intelligent body, wherein the key parameters include at least the size and equipment performance of the target intelligent body.

[0072] The target agent can be an intelligent driving device that is about to be deployed, such as a cleaning device, a passenger-carrying device, or a delivery device with autonomous driving capabilities. The dimensions of the target agent include its length, width, and height. Equipment performance includes gradeability, turning radius, and system performance. Furthermore, the target agent's model, batch, and manufacturer information may also be included.

[0073] Step 104: Determine road characterization parameters based on the image information related to the point cloud in the road information, where the road characterization parameters include at least: the slope, length, width, height, and obstacles of the current surveyed road section.

[0074] In addition, parameters such as road settlement height and obstructions can also be included.

[0075] Step 106: Perform parameter matching evaluation on the road characterization parameters of the current surveyed road section and the key parameters of the target intelligent agent.

[0076] Specifically, a digital twin model of the current surveyed road section and the target agent is constructed to map the current surveyed road section and the target agent into a virtual environment. Based on the road characterization parameters of the current surveyed road section and the key parameters of the target agent, a variety of driving behaviors are simulated. Based on the simulation results, a matching evaluation of the different parameters is performed. For example, in the digital twin model, the agent is simulated driving at different speeds on a road section with a curvature radius of r, and its steering control and driving stability are observed to obtain matching evaluation results for speed and road curvature.

[0077] Step 108: Verify the matching evaluation result based on the RGB-related image information in the road information.

[0078] Specifically, for unmatched parameter items in the matching evaluation results, or fuzzy matching parameter items, the RGB image of the corresponding road section is searched from the image database established based on RGB-related image information; based on the concrete content in the found RGB image, the matching result is analyzed and verified.

[0079] For example, the matching evaluation results may reveal that the autonomous vehicle's height prevents it from navigating a road section due to an obstruction. However, the point cloud data cannot identify the obstruction. In this case, the RGB image obtained from the image database can be used to visualize the obstruction as a tree branch. The branches can then be pruned to avoid the obstruction, providing more analytical evidence and enabling faster and more convenient analysis and solutions.

[0080] Step 110: Evaluate the suitability of the target intelligent agent for use in the current survey section and the deployment risk based on the verification results.

[0081] Finally, the verification and matching assessment results are summarized to comprehensively assess the suitability and deployment risk of the target agent for the current surveyed road section. Specifically, a table listing different parameter items can be used to summarize the suitability and deployment risk based on a scoring method.

[0082] Optionally, in this application, road information can be obtained by the following methods: Figure 2 , which may include the following steps:

[0083] Step 202: In a non-public road scenario, a point cloud image and an RGB image of the current surveyed road section are obtained respectively, and image preprocessing operations are performed.

[0084] In this application, road information can be obtained by surveying a smart handheld device, such as Figure 3 As shown, the system includes a handheld pole 301 and an intelligent host 302 located above the handheld pole 301. A solid-state laser radar 303 and a camera 304 are mounted on the outside of the intelligent host 302. The solid-state laser radar 303 is used to collect point cloud images of the surveyed road section, and the camera 304 is used to collect RGB images of the surveyed road section. The intelligent host 302 is used to receive the point cloud images collected by the solid-state laser radar 303 and the RGB images collected by the camera 304, and perform image processing to obtain the required road information.

[0085] In this application, a solid-state LiDAR with 64 or more laser lines can be used to collect point cloud images, while a camera can be used to collect RGB images. Image preprocessing operations such as classification and noise reduction are then performed on the point cloud images, while also on the RGB images, to optimize image data and improve image reliability.

[0086] Step 204: Based on the curb recognition model trained using the Faster-RCNN algorithm, the RGB curbs of the current surveyed road section are identified from the RGB image after the preprocessing operation.

[0087] Optionally, the curb recognition model trained using the Faster-RCNN algorithm is determined by:

[0088] The first step is to obtain historical RGB images of other non-public road scenes as image samples, and input the image samples into the initialized Faster-RCNN model to train the RPN network; each historical RGB image is marked with one or more curbs.

[0089] Specifically, by obtaining curb data from other non-public roads and a large amount of RGB image data collected by cameras, the feature extraction network is initialized in the ImageNe pre-trained model, and the RPN network is trained.

[0090] In the second step, the candidate boxes generated by the trained RPN network are input into the Faster-RCNN network for training.

[0091] Use the model pre-trained on ImageNet to initialize the Fast-RCNN feature extraction network, use the candidate boxes generated by the RPN network trained in the first step as input, and train a Fast-RCNN network. At this time, the parameters of each layer of the two networks are completely unshared.

[0092] The third step is to initialize a new RPN network using the network parameters trained by the Faster-RCNN network; wherein the new RPN network shares network parameters and all common convolutional layers with the Faster-RCNN network.

[0093] Initialize a new RPN network using the Fast-RCNN network parameters from step 2. At the same time, set the learning rate of the network parameters of the feature extraction network shared by the RPN and Fast-RCNN to 0. This means that the parameters unique to the RPN network are learned, while the feature extraction network is fixed. At this point, the two networks share all common convolutional layers.

[0094] The fourth step is to fine-tune the Faster-RCNN network to obtain a trained curb recognition model.

[0095] The shared network layers are fixed, the Fast-RCNN network's unique network layers are taken into account, and training continues. Based on the curb training results, the Fast-RCNN network's unique network layers are fine-tuned, so that the RPN and Fast-RCNN networks fully share parameters. The trained Fast-RCNN model can then simultaneously perform candidate box extraction and non-public road curb feature recognition. It should be understood that in this application solution, the Fast-RCNN model can include both the RPN network and the Faster-RCNN network.

[0096] The advantage of the Faster-RCNN model over other deep learning algorithms is that it introduces RPN, which directly generates candidate regions. In this application scheme, a CNN model is first used to receive the RGB image and extract the feature map. This feature map is then divided into 5×5 sliding windows, where each sliding window position maps a low-dimensional feature. These low-dimensional features are then fed into two fully connected layers, one for classification prediction and the other for regression. In the actual recognition process, 25 prior boxes of different proportions containing non-public road curb information can be set for each sliding window position, and 25 candidate regions are predicted for each sliding window position. For the classification layer, its output size can be 50, indicating the probability value of each candidate region for the possibility of containing a curb in these 50 regions, while the regression layer outputs 100 coordinate values, indicating the position of each candidate region (relative to each prior box). For each sliding window position, these two fully connected layers are shared. Therefore, RPN can be implemented using convolutional layers: first, a 5×5 convolution is performed to obtain low-dimensional features, followed by two 1×1 convolutions, used for classification and regression respectively. It should be noted that the setting of 25 prior boxes and the setting of the classification layer output size to 50 are just examples and do not limit this solution.

[0097] Therefore, the RGB image after the preprocessing operation can be input into the curb recognition model trained in the above manner to achieve curb recognition and positioning of the survey section of the non-public road.

[0098] Step 206: extracting the point cloud curb of the current surveyed road section from the point cloud image based on the road window corresponding to the set single-directional scanning line.

[0099] Optionally, in the solution of the present application, based on the road window corresponding to the set single-directional scanning line, extracting the point cloud curb of the current surveyed road section from the point cloud image may specifically include the following steps:

[0100] Step 1: Convert the point cloud image into a linear scan point cloud image comprising a series of ordered two-dimensional scan line data sets; wherein the two-dimensional scan lines are perpendicular to the survey travel direction.

[0101] Solid-state laser radar mainly performs linear scanning, and the scan points obtained are arranged according to the scan lines. In the road environment, the top is mostly the sky, and the data obtained by the vehicle-mounted laser scanning system is distributed in a strip-shaped, non-closed manner. The vehicle-mounted laser scanner mostly uses a panoramic scan with a 360-degree field of view. The arrangement of the scan lines extracted according to the GPS time or scanning angle of the point cloud is basically along the direction of travel of the vehicle. Each scan line is equivalent to a cross section of the road. The discretely distributed point cloud in the point cloud image is converted into a series of ordered two-dimensional scan line data sets, that is, Figure 4a The linear scan point cloud shown.

[0102] Step 2: extract the scanning line cross-section corresponding to each two-dimensional scanning line from the linear scanning point cloud image as a road window.

[0103] From the linear scan point cloud, extract the scan line cross-section corresponding to each two-dimensional scan line as the road window, and refer to Figure 4b As shown in the figure, different targets (here referring to targets such as roads, curbs, and obstructions in the point cloud image) show different spatial distributions and geometric shapes on the same scan line. At the same time, adjacent scan lines have similar spatial distributions. From the spatial distribution law of the point cloud scan line, the ground is relatively flat and the elevation value is relatively small and does not change much. It is generally distributed horizontally on the scan line and can be approximately regarded as a smooth and continuous horizontal straight line.

[0104] In step 3, based on the discrete point clouds on the left and right sides of each road window, the curb boundaries on the left and right sides are estimated, where the curb boundaries include the curb boundary position and height.

[0105] Still refer to Figure 4bAs shown, for each road window, the discrete point cloud on the left is counted and clustered to obtain the left curb boundary corresponding to the left point cloud. The discrete point cloud on the right is counted and clustered to obtain the right curb boundary corresponding to the right point cloud. This boundary does not need to be precise; it only needs to be estimated based on the discrete distance calculation between the point clouds to form a virtual center point of the curb boundary. Through coordinate calculation and matching, fuzzy calculation is performed to cluster the point clouds within 10 cm to the left and right of the center point and 50 cm in height. This determines the location and height of the curb boundary, resulting in the left and right curb boundaries.

[0106] In step 4, the curb boundaries of each road window are summarized to obtain the point cloud curb of the current survey section.

[0107] Step 208: Based on the confidence evaluation index, perform confidence evaluation on the RGB curb and the point cloud curb.

[0108] For RGB curbs and point cloud curbs, the confidence evaluation method can be used to select the same or different confidence evaluation indexes to perform confidence evaluation separately.

[0109] Step 210: If the confidence evaluation results of the RGB curb and the confidence evaluation results of the point cloud curb are both greater than the set first threshold, it is determined that the survey is reliable, and the road information of the current survey section is determined based on the point cloud curb and the RGB curb.

[0110] In the present application, the first threshold may be flexibly set based on empirical data or business requirements. For example, the first threshold may be 80%, 85%, 90%, or 95%.

[0111] Optionally, when determining the road information of the current survey section based on the point cloud curb and the RGB curb, the RGB image can be cut based on the RGB curb, wherein the RGB image retained after cutting includes the curb image area and the road surface image area between the curbs; and, the point cloud image can be cut based on the point cloud curb, wherein the point cloud image retained after cutting includes the curb image area and the road surface image area between the curbs; thereafter, the RGB image retained after cutting and the point cloud image retained after cutting are used as the road information of the current survey section.

[0112] When the point cloud image is cut based on the point cloud curb, the image area outside the point cloud curb in the point cloud image is cut and removed, and the image area in the point cloud image whose distance from the road surface is greater than a set second threshold is cut and removed. Figure 4b As shown, for example, if the second threshold is set to 5m, the area below the red dotted box is cut out and the required road information is retained.

[0113] Step 212: If at least one of the confidence evaluation result of the RGB curb and the confidence evaluation result of the point cloud curb is not greater than the set first threshold, it is determined that the current survey is unreliable and the above survey is repeated.

[0114] Taking the confidence evaluation index of 95% as an example, if the confidence evaluation result of the RGB curb is greater than 95%, but the confidence evaluation result of the point cloud curb is less than 95%; or, the confidence evaluation result of the RGB curb is less than 95%, but the confidence evaluation result of the point cloud curb is greater than 95%; or, the confidence evaluation result of the RGB curb is less than 95%, but the confidence evaluation result of the point cloud curb is less than 95%, then it is determined that the survey data is insufficient and unreliable, and the above survey needs to be re-executed.

[0115] It should be noted that based on Figure 3 In the handheld device shown, a corresponding radar indicator light is located below the solid-state laser radar 303, and a corresponding camera indicator light is located below the camera 304. When the confidence assessment results indicate that the survey is reliable, both the radar and camera indicators on the handheld device illuminate green. When the confidence assessment results indicate that the survey is unreliable, the corresponding indicators illuminate red. These indicators provide a timely and convenient reminder to the handheld user regarding the reliability of the survey results, allowing for prompt re-survey or continued survey of the next road section.

[0116] Reference Figure 5a , which is a schematic diagram of the survey process of a non-public road provided in an embodiment of the present application.

[0117] The user turns on the survey switch of the handheld device, and the solid-state lidar and camera simultaneously collect and process images.

[0118] For the solid-state lidar side: acquire point cloud images through the lidar, perform image classification and noise reduction preprocessing on the point cloud images, determine the curbs in the point cloud, and perform a confidence evaluation on the determined curbs. If the confidence level is greater than 90%, segment and extract the point cloud curbs; otherwise, jump back to the step of acquiring the point cloud images.

[0119] On the camera side: RGB images are acquired through the camera, image classification and noise reduction are performed on the RGB images, curbs in the images are identified and located, and the confidence level of the identified curbs is evaluated. If the confidence level is greater than 95%, the curbs in the image are segmented and extracted. Otherwise, the process jumps back to the step of acquiring the RGB image.

[0120] This technical solution, through RGB and point cloud image recognition and segmentation, can obtain information related only to roads and curbs, enabling rapid offline processing with low equipment requirements, and directly filtering out other interfering images and point clouds. At the same time, the introduction of a confidence evaluation index enables offline evaluation of the acquisition operation, ensuring the reliability of the data source required for the acquisition.

[0121] Furthermore, in the present application scheme, after surveying non-public roads and obtaining road information of the surveyed sections, the road information can also be used, combined with the product parameter information of the intelligent body, to automatically and intelligently evaluate the feasibility of deploying different intelligent bodies on the surveyed non-public roads, to determine which intelligent bodies are suitable for deployment on the surveyed non-public roads, which are not suitable for deployment, and the risks of deployment, etc., thereby avoiding the dependence and uncertainty of manual evaluation and improving evaluation efficiency and accuracy.

[0122] Reference Figure 5b , which is a schematic diagram of the survey and evaluation process of a non-public road provided in an embodiment of the present application.

[0123] Turn on the handheld device's survey function to extract road characterization parameters based on the point cloud image. Simultaneously, establish an image database based on the RGB image. Turn on the product addition function for the product adding device (this product addition device can be a third-party communication device, such as a mobile phone, or the handheld device itself, meaning it has product addition functionality) to obtain key product parameters. These road characterization parameters and key product parameters are then evaluated in the matching evaluation backend. The evaluation results are then output and verified against the RGB images in the image database.

[0124] The evaluation scheme for this part does not require field deployment tests of target intelligent agents. It only requires the advance summary or real-time input of key parameters of different target intelligent agents, which can meet the rapid switching evaluation of different intelligent agent products. At the same time, it can quickly evaluate the adaptability and deployment risks of roads and intelligent agent products through comparative evaluation of characterization parameters.

[0125] It should be noted that for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that this application is not limited by the order of the actions described, because according to this application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by this application.

[0126] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0127] Figure 6 FIG. 1 shows a structural block diagram of a survey device for non-public roads provided by an embodiment of the present application. Figure 6 As shown. The matching evaluation device 600 for deploying a target intelligent body to a current survey section of the present embodiment may include an acquisition module 601, a determination module 602, a matching module 603, a verification module 604 and an evaluation module 605. Among them, the acquisition module 601 is used to acquire the road information of the current survey section and the key parameters of the target intelligent body, and the key parameters include at least: the size and equipment performance of the target intelligent body; the determination module 602 is used to determine the road characterization parameters based on the image information related to the point cloud in the road information, and the road characterization parameters include at least: the slope, length, width, height and obstacles of the current survey section; the matching module 603 is used to perform parameter matching evaluation on the road characterization parameters of the current survey section and the key parameters of the target intelligent body; the verification module 604 is used to verify the matching evaluation results based on the RGB-related image information in the road information; the evaluation module 605 is used to evaluate the adaptability and deployment risk of the target intelligent body deployed in the current survey section according to the verification results.

[0128] It should be noted that part or all of the survey device for non-public roads in this embodiment can be an application located in the local terminal, or can also be a functional unit such as a plug-in or software development kit (SDK) set in the application located in the local terminal, or can also be a processing engine located in the network side server, or can also be a distributed system located on the network side. This embodiment does not specifically limit this.

[0129] It is understandable that the application may be a native program (nativeApp) installed on the local terminal, or may be a webpage program (webApp) of a browser on the local terminal, which is not limited in this embodiment.

[0130] Optionally, in a possible implementation of this embodiment, when performing parameter matching evaluation on the road characterization parameters of the current surveyed road section and the key parameters of the target agent, the matching module 603 is specifically configured to:

[0131] Construct a digital twin model of the current surveyed road section and the target intelligent agent to map the current surveyed road section and the target intelligent agent into a virtual environment; simulate a variety of different driving behaviors based on the road characterization parameters of the current surveyed road section and the key parameters of the target intelligent agent; and perform matching evaluation of different parameters based on the simulation results.

[0132] Optionally, in a possible implementation of this embodiment, when verifying the matching evaluation result based on the RGB-related image information in the road information, the verification module 604 is specifically configured to:

[0133] For unmatched parameter items in the matching evaluation results, or parameter items with ambiguous matching, the RGB image of the corresponding road section is searched from the image database established based on RGB-related image information; based on the concrete content in the found RGB image, the matching result is analyzed and verified.

[0134] Optionally, in a possible implementation of this embodiment, when acquiring the road information of the current surveyed road section, the acquisition module 601 is specifically configured to:

[0135] In a non-public road scenario, a point cloud image and an RGB image of the current surveyed road section are obtained respectively, and image preprocessing operations are performed; based on a curb recognition model trained using the Faster-RCNN algorithm, the RGB curb of the current surveyed road section is identified from the RGB image after the preprocessing operation; based on a road window corresponding to a set single-direction scan line, the point cloud curb of the current surveyed road section is extracted from the point cloud image; different confidence evaluation indices are used to perform confidence evaluation on the RGB curb and the point cloud curb respectively; if the confidence evaluation results of the RGB curb and the confidence evaluation results of the point cloud curb are both greater than a set first threshold, it is determined that this survey is reliable, and road information of the current surveyed road section is determined based on the point cloud curb and the RGB curb; if at least one of the confidence evaluation results of the RGB curb and the confidence evaluation results of the point cloud curb is not greater than the set first threshold, it is determined that this survey is unreliable, and the above-mentioned survey is performed again.

[0136] It should be noted that this optional solution is based on a device that integrates surveying and evaluation, that is, the matching evaluation device has a surveying function, so road information can be obtained through surveying means.

[0137] Optionally, in a possible implementation of this embodiment, when the acquisition module 601 extracts the point cloud curb of the current surveyed road section from the point cloud image based on the road window corresponding to the set single-directional scan line, it is specifically configured to:

[0138] The point cloud image is converted into a linear scan point cloud map comprising a series of ordered two-dimensional scan line data sets; wherein the two-dimensional scan lines are perpendicular to the survey travel direction; a scan line cross-section corresponding to each two-dimensional scan line is extracted from the linear scan point cloud map as a road window; and the curb boundaries on the left and right sides are estimated based on the discrete point clouds on the left and right sides of each road window, wherein the curb boundaries include the curb boundary position and height; and the curb boundaries of each road window are aggregated to obtain a point cloud curb of the current survey section.

[0139] Optionally, in a possible implementation of this embodiment, when the acquisition module 601 determines the road information of the current survey section based on the point cloud curb and the RGB curb, it is specifically used to: cut the RGB image based on the RGB curb, wherein the RGB image retained after cutting includes the curb image area and the road surface image area between the curbs; cut the point cloud image based on the point cloud curb, wherein the point cloud image retained after cutting includes the curb image area and the road surface image area between the curbs; and use the RGB image retained after cutting and the point cloud image retained after cutting as the road information of the current survey section.

[0140] Optionally, in a possible implementation of this embodiment, when the acquisition module 601 cuts the point cloud image based on the point cloud curb, it is specifically configured to:

[0141] The image area outside the point cloud curb in the point cloud image is cut off and removed, and the image area within the point cloud image that is set to a second threshold away from the road surface is cut off and removed.

[0142] Reference Figure 7 As shown, the present application also provides a matching evaluation system 700 for placing a target intelligent agent on a currently surveyed road section, including: a survey device 701 and an evaluation device 702.

[0143] Among them, the survey device 701 can be Figure 3The handheld device shown is used to perform road section surveys, specifically: in a non-public road scenario, a point cloud image and an RGB image of the current surveyed road section are respectively obtained, and image preprocessing operations are performed; based on a curb recognition model trained using the Faster-RCNN algorithm, the RGB curb of the current surveyed road section is identified from the RGB image after the preprocessing operation; based on a road window corresponding to a set single-directional scan line, the point cloud curb of the current surveyed road section is extracted from the point cloud image; based on a confidence evaluation index, the confidence of the RGB curb and the point cloud curb are respectively evaluated; if the confidence evaluation results of the RGB curb and the confidence evaluation results of the point cloud curb are both greater than a set first threshold, the current survey is determined to be reliable, and road information of the current surveyed road section is determined based on the point cloud curb and the RGB curb; if at least one of the confidence evaluation results of the RGB curb and the confidence evaluation results of the point cloud curb is not greater than the set first threshold, the current survey is determined to be unreliable, and the above-mentioned survey is performed again.

[0144] The evaluation device 702 can be a background server, which is used to perform a matching evaluation between the surveyed road and the target intelligent body. Specifically, the key parameters of the target intelligent body are obtained, and the key parameters include at least: the size and equipment performance of the target intelligent body; the road characterization parameters are determined based on the image information related to the point cloud in the road information, and the road characterization parameters include at least: the slope, length, width, height and obstacles of the current surveyed road section; the road characterization parameters of the current surveyed road section are evaluated for parameter matching with the key parameters of the target intelligent body; the matching evaluation results are verified based on the RGB-related image information in the road information; and the adaptability and deployment risk of the target intelligent body for use in the current surveyed road section are evaluated based on the verification results.

[0145] Among them, the background server can be a cloud background or a distributed background, or an ordinary server, and this application does not limit this.

[0146] In this embodiment, road information for the currently surveyed section and key parameters of the target agent are obtained. Road characterization parameters are determined based on the point cloud-related image information in the road information. A parameter matching assessment is performed between the road characterization parameters of the currently surveyed section and the key parameters of the target agent. The matching assessment results are verified based on the RGB-related image information in the road information. Based on the verification results, the suitability and deployment risk of the target agent for use in the currently surveyed section are assessed. This eliminates the need for manual evaluation expertise and enables automated intelligent matching assessment, reducing evaluation costs and time consumption while achieving high evaluation accuracy.

[0147] One embodiment of the present application provides a computer-readable storage medium, wherein the storage medium stores at least one instruction, and the at least one instruction is loaded and executed by a processor to implement the blind spot detection method as described above.

[0148] An embodiment of the present application provides an electronic device, which includes a processor and a memory, wherein the memory stores at least one instruction, and the instruction is loaded and executed by the processor to implement the blind spot detection method as described above.

[0149] In the technical solution of this application, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0150] Figure 8 A schematic block diagram of an example electronic device 800 that can be used to implement an embodiment of the present application is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or claimed herein.

[0151] like Figure 8 As shown, the electronic device 800 includes a computing unit 801, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the electronic device 800 can also be stored. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0152] Multiple components in the electronic device 800 are connected to the I / O interface 805, including an input unit 806, such as a keyboard, a mouse, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a magnetic disk, an optical disk, etc.; and a communication unit 809, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 809 allows the electronic device 800 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0153] The computing unit 801 can be a variety of general-purpose and / or specialized processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units that run machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The computing unit 801 performs the various methods and processes described above, such as the blind spot detection method. For example, in some embodiments, the blind spot detection method can be implemented as a computer software program that is tangibly contained in a machine-readable medium, such as the storage unit 808. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 800 via the ROM 802 and / or the communication unit 809. When the computer program is loaded into the RAM 803 and executed by the computing unit 801, one or more steps of the blind spot detection method described above can be performed. Alternatively, in other embodiments, the computing unit 801 can be configured to perform the blind spot detection method by any other appropriate means (e.g., by means of firmware).

[0154] Various embodiments of the systems and techniques described above can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, at least one input device, and at least one output device.

[0155] The program code for implementing the methods of the present application can be written in any combination of one or more programming languages. Such program code can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the program code is executed by the processor or controller, the functions / operations specified in the flow charts and / or block diagrams are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0156] In the context of the present application, a machine-readable medium can be a tangible medium that can contain or store a program for use by an instruction execution system, device or equipment or used in combination with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0157] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0158] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0159] A computer system may include a client and a server. The client and server are generally remote from each other and typically interact through a communication network. The client-server relationship arises through computer programs running on the respective computers and having a client-server relationship with each other. The server may be a cloud server, a server in a distributed system, or a server integrated with a blockchain.

[0160] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this application can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this application can be achieved. This is not a limitation herein.

[0161] The above specific embodiments do not constitute a limitation on the scope of protection of this application. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application shall be included within the scope of protection of this application.

Claims

1. A matching evaluation method for placing a target agent on a current survey section, characterized in that: include: Obtaining road information of the current surveyed road section and key parameters of the target intelligent body, wherein the key parameters include at least the size and equipment performance of the target intelligent body; Determining road characterization parameters based on image information related to the point cloud in the road information, wherein the road characterization parameters include at least: the slope, length, width, height, and obstacles of the current surveyed road section; Perform parameter matching evaluation on the road characterization parameters of the current surveyed road section and the key parameters of the target intelligent agent; Verifying the matching evaluation result based on the RGB-related image information in the road information; Based on the verification results, the suitability and deployment risk of the target intelligent agent for use in the current survey section are evaluated.

2. The method according to claim 1, wherein Perform parameter matching evaluation on the road characterization parameters of the current surveyed road section and the key parameters of the target intelligent agent, specifically including: Constructing a digital twin model of the current surveyed road section and the target intelligent agent to map the current surveyed road section and the target intelligent agent into a virtual environment; Based on the road characterization parameters of the current surveyed road section and the key parameters of the target intelligent agent, simulating a variety of different driving behaviors; The matching evaluation of different parameters is carried out based on the simulation results.

3. The method according to claim 1, wherein Verify the matching evaluation result based on the RGB-related image information in the road information, specifically including: For the parameter items that do not match or are ambiguous in the matching evaluation results, the RGB image of the corresponding road section is searched from the image database established based on RGB-related image information; Analyze and verify the matching results based on the concrete content in the found RGB image.

4. The method according to claim 1, wherein Get the road information of the current survey section, including: In non-public road scenarios, obtain the point cloud image and RGB image of the current survey section and perform image preprocessing operations; Based on the curb recognition model trained by the Faster-RCNN algorithm, the RGB curb of the current surveyed road section is identified from the RGB image after preprocessing operation; Extracting a point cloud curb of the current surveyed road section from the point cloud image based on a road window corresponding to a set single-directional scanning line; Using different confidence evaluation indexes respectively, performing confidence evaluation on the RGB curb and the point cloud curb; If the confidence evaluation results of the RGB curb and the confidence evaluation results of the point cloud curb are both greater than the set first threshold, the survey is determined to be reliable, and the road information of the current surveyed section is determined based on the point cloud curb and the RGB curb; If at least one of the confidence evaluation result of the RGB curb and the confidence evaluation result of the point cloud curb is not greater than the set first threshold, it is determined that the current survey is unreliable and the above survey is performed again.

5. The method according to claim 4, wherein Extracting the point cloud curb of the current surveyed road section from the point cloud image based on the road window corresponding to the set single-directional scanning line, specifically including: Converting the point cloud image into a linear scan point cloud map comprising a series of ordered two-dimensional scan line data sets; wherein the two-dimensional scan lines are perpendicular to the survey travel direction; Extracting a scan line cross-section corresponding to each two-dimensional scan line from the linear scan point cloud map as a road window; Estimate the curb boundaries on the left and right sides based on the discrete point clouds on the left and right sides of each road window, respectively, wherein the curb boundaries include the curb boundary position and height; The curb boundaries of each road window are summarized to obtain the point cloud curb of the current survey section.

6. The method according to claim 5, wherein Determining the road information of the current surveyed road section based on the point cloud curb and the RGB curb specifically includes: Cutting the RGB image based on the RGB curb, wherein the RGB image retained after cutting includes a curb image area and a road surface image area between the curbs; Cutting the point cloud image based on the point cloud curb, wherein the point cloud image retained after cutting includes a curb image area and a road surface image area between the curbs; The RGB image and the point cloud image retained after cutting are used as the road information of the current survey section.

7. The method according to claim 6, wherein Cutting the point cloud image based on the point cloud curb specifically includes: The image area outside the point cloud curb in the point cloud image is cut off and removed, and the image area within the point cloud image that is set to a second threshold away from the road surface is cut off and removed.

8. A matching evaluation device for placing a target intelligent agent on a currently surveyed road section, characterized in that: include: An acquisition module is used to acquire road information of the current surveyed road section and key parameters of the target intelligent body, wherein the key parameters include at least the size and equipment performance of the target intelligent body; a determination module, configured to determine road characterization parameters based on image information related to the point cloud in the road information, the road characterization parameters including at least: a slope, length, width, height, and obstacles of a currently surveyed road section; A matching module, configured to perform parameter matching evaluation between the road characterization parameters of the current surveyed road section and the key parameters of the target intelligent agent; a verification module, configured to verify the matching evaluation result based on the RGB-related image information in the road information; The evaluation module is used to evaluate the suitability and deployment risk of the target intelligent agent for use in the current survey section based on the verification results.

9. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 7.

10. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 7.

Images