Information validity determination method and device, equipment and storage medium

Abstract

The invention provides an information validity determination method, and relates to the technical field of artificial intelligence, in particular to the technical fields of map navigation, automatic driving, intelligent traffic and the like. The method comprises the steps that images in a first image set and images in a second image set are paired according to the similarity between image features, at least one image pair is obtained, and the collection time of the second image set is earlier than that of the first image set; determining a target object in the image pair according to a matching result of the image pixel points in the image pair; according to the feature similarity between the target object in the first image and the target object in the second image in the image pair, determining the object category of the target object in the image pair; the object category of the target object in the at least one image pair is counted, state information of the target object is determined according to a statistical result, and the state information comprises an effective state, a failure state and an indefinite state. The method improves the accuracy of the validity information.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technologies, specifically to technologies such as map navigation, autonomous driving, and intelligent transportation, and particularly to a method, apparatus, device, and storage medium for determining information validity. Background Art

[0002] With the popularization of mobile electronic map applications, electronic maps are increasingly used by users. POI (Point of Interest) represents a specific location point on the map that is of practical value or attraction to users. It is a core component of map data and can help users quickly locate, search for, and navigate to the target location. Summary of the Invention

[0003] The present disclosure provides a method, apparatus, device, and storage medium for determining information validity.

[0004] According to a first aspect of the present disclosure, there is provided a method for determining information validity, including: pairing images in a first image set and a second image set according to the similarity between image features to obtain at least one image pair, where the acquisition time of the second image set is earlier than that of the first image set; determining a target object in the image pair according to the matching result of the image pixel points in the image pair; determining the object category of the target object in the image pair according to the feature similarity between the target object in the first image and the target object in the second image in the image pair; and statistically analyzing the object categories of the target object in at least one image pair, and determining the status information of the target object according to the statistical result, where the status information includes: valid status, invalid status, and unclear status.

[0005] According to a second aspect of the present disclosure, there is provided a device for determining information validity, including: an image pair determination module configured to pair images in a first image set and a second image set according to the similarity between image features to obtain at least one image pair, where the acquisition time of the second image set is earlier than that of the first image set; a target object determination module configured to determine a target object in the image pair according to the matching result of the image pixel points in the image pair; an object category determination module configured to determine the object category of the target object in the image pair according to the feature similarity between the target object in the first image and the target object in the second image in the image pair; and a status information determination module configured to statistically analyze the object categories of the target object in at least one image pair, and determine the status information of the target object according to the statistical result, where the status information includes: valid status, invalid status, and unclear status.

[0006] According to a third aspect of the present disclosure, there is provided an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein, the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method described in any implementation manner of the first aspect.

[0007] According to a fourth aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium storing computer instructions for causing a computer to execute the method described in any implementation manner of the first aspect.

[0008] According to a fifth aspect of the present disclosure, there is provided a computer program product including a computer program, and the computer program implements the method described in any implementation manner of the first aspect when executed by a processor.

[0009] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it used to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The drawings are used to better understand the solution and do not constitute a limitation to the present disclosure. Among them:

[0011] Figure 1 is an exemplary system architecture diagram to which the present disclosure can be applied;

[0012] Figure 2 is a flowchart of an embodiment of a method for determining the validity of information according to the present disclosure;

[0013] Figure 3 is a flowchart of another embodiment of a method for determining the validity of information according to the present disclosure;

[0014] Figure 4 is a flowchart of yet another embodiment of a method for determining the validity of information according to the present disclosure;

[0015] Figure 5 is Figure 4 a flowchart of a step of determining the status information of a target object in

[0016] Figure 6 is a flowchart of still another embodiment of a method for determining the validity of information according to the present disclosure;

[0017] Figure 7 is an application flowchart of a method for determining the validity of information according to the present disclosure;

[0018] Figure 8It is a schematic structural diagram of an embodiment of an apparatus for determining the validity of information according to the present disclosure;

[0019] Figure 9 It is a block diagram of an electronic device for implementing the method for determining the validity of information in an embodiment of the present disclosure. Detailed implementation manners

[0020] The exemplary embodiments of the present disclosure will be described below with reference to the accompanying drawings. Various details of the embodiments of the present disclosure are included to facilitate understanding, and they should be considered merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, descriptions of well-known functions and structures are omitted in the following description for clarity and conciseness.

[0021] It should be noted that, without conflict, the embodiments in the present disclosure and the features in the embodiments can be combined with each other. The present disclosure will be described in detail below with reference to the drawings and in conjunction with the embodiments.

[0022] Figure 1 An exemplary system architecture 100 is shown, which can apply the embodiments of the method for determining the validity of information or the apparatus for determining the validity of information according to the present disclosure.

[0023] As Figure 1 shown, the system architecture 100 may include terminal devices 101, 102, 103, 104, a network 105, and a server 106. The network 105 is used as a medium to provide a communication link between the terminal devices 101, 102, 103, 104 and the server 106. The network 105 may include various connection types, such as wired, wireless communication links, or fiber optic cables, etc.

[0024] Users can use the terminal devices 101, 102, 103, 104 to interact with the server 106 through the network 105 to receive or send information, etc. Various client applications may be installed on the terminal devices 101, 102, 103, 104.

[0025] The terminal devices 101, 102, 103, 104 may be hardware or software. When the terminal devices 101, 102, 103, 104 are hardware, they may be various electronic devices, including but not limited to smart phones, tablet computers, laptop portable computers, and desktop computers, etc. When the terminal devices 101, 102, 103, 104 are software, they may be installed in the above-mentioned electronic devices. They may be implemented as multiple software or software modules, or may be implemented as a single software or software module. No specific limitation is made herein.

[0026] Server 106 can provide various services. For example, server 106 can analyze and process the images in the first image set and the second image set obtained from the terminal devices 101, 102, 103, and 104, and generate a processing result (such as the status information of the target object).

[0027] It should be noted that server 106 can be hardware or software. When server 106 is hardware, it can be implemented as a distributed server cluster composed of multiple servers or as a single server. When server 106 is software, it can be implemented as multiple software or software modules (such as those used to provide distributed services) or as a single software or software module. No specific limitation is made here.

[0028] It should be noted that the method for determining the information validity provided in the embodiments of the present disclosure is generally executed by server 106. Correspondingly, the device for determining the information validity is generally set in server 106.

[0029] It should be understood that Figure 1 the numbers of the terminal devices, networks, and servers in

[0030] Continue to refer to Figure 2 , which shows a flow 200 of an embodiment of the method for determining the information validity according to the present disclosure. The method for determining the information validity includes the following steps:

[0031] Step 201, pair the images in the first image set and the second image set according to the similarity between the image features to obtain at least one image pair.

[0032] In this embodiment, the execution subject of the method for determining the information validity (such as Figure 1 the server 105 shown) will pair the images in the first image set and the second image set according to the similarity between the image features to obtain at least one image pair, where the acquisition time of the second image set is earlier than that of the first image set.

[0033] Here, the road images can be collected in real time through multi-modal sensors carried by the vehicle (including high-precision GPS, IMU, and 360° surround-view cameras), so as to obtain the latest collected image set, which is called the new image set (i.e., the first image set). The new image set here can include consecutive images collected for one road, or can include consecutive images collected for multiple roads. This embodiment does not make specific limitations on this. If the new image set includes consecutive images collected for multiple roads, the above-mentioned execution entity will first divide the new image set into multiple subsets according to the roads, and each subset corresponds to the images of one road, and then obtain the road identifiers of each road. Here, a link will be pre-assigned to each road, and the links of each road are different. Therefore, the link is the ID (Identity document) of this road. Therefore, the above-mentioned execution entity will obtain the links of each road and use the link as the identification information of this road.

[0034] After that, the above-mentioned execution entity will obtain the images corresponding to the road identifiers from the historical image set (historically collected images). If there are multiple road identifiers, the historical images corresponding to each road identifier will be obtained respectively, so as to obtain multiple historical image subsets.

[0035] Considering that some roads are two-way roads but are named with the same link, it is necessary to further determine the direction of the link. Here, the pose of the new image can be estimated first to determine the vehicle heading angle corresponding to the new image, and then the coordinate information of the new image can be obtained. Further, the vehicle heading angle and distance (coordinate offset) are used to recall the historical images in the same direction as the new image from the historical image set. For example, recall the historical images with a vehicle heading angle deviation within ±15° and a coordinate offset within 10 meters, so as to obtain the historical image set on the same road and in the same direction as the new image set, which can also be called the old image set (i.e., the second image set).

[0036] Finally, the above-mentioned execution entity will pair the images in the new image set with the images in the old image set according to the similarity between the image features in the new image set and the image features in the old image set, so as to obtain at least one image pair.

[0037] Here, the above-mentioned execution entity can first determine at least one initial image pair according to a pre-constructed graph neural network. The graph neural network is a graph neural network (GNN) with links as nodes and the topological relationships between new images, the topological relationships between old images, and the topological relationships between new and old images as edges. Through the graph neural network, K nearest neighbor image pairs that satisfy the distance constraint (e.g., within 50 meters) can be determined. For example, K can take the value of 50. Here, the topological relationship can include temporal relationship, spatial relationship, etc. Then, for each of the K image pairs, the above-mentioned execution entity will use a visual localization algorithm to encode the new and old images in the image pair into a series of feature values respectively, and determine the similarity between the new and old images in the image pair by comparing the feature value sequences of the two images, so as to obtain at least one high-similarity image pair.

[0038] Step 202: Determine the target object in the image pair according to the matching result of the image pixel points in the image pair.

[0039] In this embodiment, for each of the at least one image pair (i.e., the high-similarity image pair), the above-mentioned execution entity will determine the target object in the image pair according to the matching result of the image pixel points of the new and old images in the image pair. Here, the object in the image will be determined first. The object is a location point in the real world, which can be a natural landscape, a commercial place, a public facility, etc. For example, the object can be a store sign, a mailbox, a bus stop sign, etc. The target object is the object with high similarity in the new and old images. That is, the above-mentioned execution entity can determine whether the object in the image pair is the target object according to the matching result of the image pixel points in the image pair.

[0040] Here, the above-mentioned execution entity can first identify the new and old images in the image pair respectively to identify and locate the objects in the new and old images. Here, taking the object as a sign for illustration, that is, the above-mentioned execution entity will first identify and locate the signs in the new and old images to determine the positions of the signs in the new and old images, and mark the signs in the image, so as to obtain the detection frame corresponding to the sign.

[0041] Then, use the local feature matching algorithm to identify the combination of similar pixel points in the new and old images. After that, determine the similar pixel points included in the detection frame corresponding to the sign in the new image, and calculate the density of the similar pixel points in the detection frame. If the density is greater than the preset density threshold (e.g., 15), it is determined that the sign is a suspected similar sign, that is, it is determined that the sign is the target object.

[0042] Step 203: Determine the object category of the target object in the image pair according to the feature similarity between the target object in the first image and the target object in the second image in the image pair.

[0043] In this embodiment, the above-mentioned execution entity determines the object category of the target object in the image pair according to the feature similarity between the target object in the first image and the target object in the second image in the image pair.

[0044] The above-mentioned execution entity constructs a 256-dimensional deep feature encoder to perform feature encoding on the target objects in the new and old images, calculates the cosine similarity between the target objects in the new and old images according to the feature encoding, and designs a dynamic threshold decision mechanism, so as to determine the object category of the target object according to the dynamic threshold decision mechanism. The object categories here can include: high-confidence valid objects and low-confidence invalid objects. A high-confidence valid object indicates that the target object is an object with high confidence, that is, it is probably a valid object. A low-confidence invalid object indicates that the target object is an object with low confidence, that is, it is probably an invalid object.

[0045] Specifically, if the cosine similarity is greater than or equal to the first similarity threshold (for example, 0.95), it is determined that the object category of the target object in the image pair is a high-confidence valid object, that is, the target object is valid; if the cosine similarity is less than the second similarity threshold (for example, 0.85), it is determined that the object category of the target object in the image pair is a low-confidence invalid object, that is, the target object has failed; if the cosine similarity is greater than the second similarity threshold and less than the first similarity threshold, that is, the cosine similarity is between the first similarity threshold and the second similarity threshold. At this time, the object category of the target object cannot be determined, and it needs to be further verified to determine the object category of the target object according to the verification result.

[0046] Step 204, count the object categories of the target object in at least one image pair, and determine the status information of the target object according to the statistical result.

[0047] In this embodiment, the above-mentioned execution entity counts the object categories of the target object in at least one image pair, and determines the status information of the target object according to the statistical result. The status information includes: valid status, invalid status, and unclear status.

[0048] That is, for each image pair, the above-mentioned execution entity will determine the object category of the target object in the image pair through steps 202-203. After steps 202-203 have been executed for all image pairs, the above-mentioned execution entity will count the object category information of the target object in all image pairs.

[0049] Since both the first image set and the second image set are consecutive images of the road collected, there will be overlapping phenomena in the consecutive images of the first image set and the consecutive images of the second image set. Therefore, there may also be overlapping among multiple image pairs. For example, object 1 exists in the first image pair and also in the second image pair; for another example, object 2 exists in the first image pair, also in the second image pair, and also in the third image pair.

[0050] Therefore, the above-mentioned execution entity will count the object category information of the target object in all image pairs and determine the status information of the target object according to the statistical results. For example, if the target object is determined to be a high-confidence valid object multiple times in all image pairs, then the status information of the target object can be determined to be the valid status; if the target object is determined to be a low-confidence invalid object multiple times in all image pairs, then the status information of the target object can be determined to be the invalid status; if the target object is determined to be a high-confidence valid object or a low-confidence invalid object only once in all image pairs, it is considered that the current result is not representative. At this time, the status of the target object cannot be determined as valid or invalid, that is, the status of the target object is in an unclear state at this time, and a secondary acquisition task needs to be performed to determine whether the status of the target object is valid or invalid.

[0051] As an example, taking a signboard as the object for illustration, if the statistical results show that signboard 1 is determined to be a high-confidence valid object 4 times in all image pairs, signboard 2 is determined to be a low-confidence invalid object 3 times in all image pairs, and signboard 3 is determined to be a high-confidence valid object 1 time in all image pairs, then the final status of signboard 1 is determined to be valid (that is, signboard 1 still exists), the final status of signboard 2 is determined to be invalid (that is, it does not exist, for example, it has been demolished), and the status of signboard 3 is in an unclear state, and a secondary acquisition task needs to be performed to determine whether the status of the target signboard 3 is valid or invalid.

[0052] The method for determining information validity provided by the embodiments of the present disclosure first pairs the images in the first image set and the second image set according to the similarity between image features to obtain at least one image pair; then determines the target object in the image pair according to the matching result of the image pixel points in the image pair; then determines the object category of the target object in the image pair according to the feature similarity between the target object in the first image and the target object in the second image in the image pair; finally, counts the object categories of the target object in at least one image pair, and determines the status information of the target object according to the statistical result. In the method for determining information validity in this embodiment, the method first pairs the continuous new and old image sets on the same road to obtain multiple image pairs, and then determines whether the final status of the target object is valid, invalid, or uncertain according to the object category of the target object in multiple image pairs, so as to determine the validity information of the target object based on the continuity information of the road, and determine the final status through the object category information of the target object in at least one image pair, avoiding the inaccurate validity information caused by the limitation of a single image and improving the accuracy of the validity information.

[0053] In addition, in the technical solutions involved in the present disclosure, the acquisition, storage, use, processing, transportation, provision, and disclosure of the user's personal information involved (such as the first image set and the second image set involved in the present disclosure) comply with the provisions of relevant laws and regulations and do not violate public order and good customs.

[0054] Continue to refer to Figure 3 , Figure 3 shows a flow 300 of another embodiment of the method for determining information validity according to the present disclosure. The method for determining information validity includes the following steps:

[0055] Step 301, pair the images in the first image set and the second image set according to the similarity between image features to obtain at least one image pair.

[0056] In this embodiment, the execution subject of the method for determining information validity (such as Figure 1 the server 105 shown) will pre-construct a graph neural network, which is a graph neural network with road links as nodes and the topological relationships between the first images, the topological relationships between the second images, and the topological relationships between the first images and the second images as edges. The similarity between the graphs is calculated through a message passing mechanism, so as to output K nearest neighbor image pairs that meet the constraint, that is, output K nearest neighbor image pairs that meet the distance constraint (such as ), for example, K can take a value of 50.

[0057] Then, for each of the K image pairs, the above-mentioned execution entity will use a visual localization algorithm to encode the first image and the second image in the image pair into a series of feature values respectively, and determine the similarity between the first image and the second image in the image pair by comparing the feature value sequences of the two images, so as to obtain at least one high-similarity image pair.

[0058] Specifically, the visual localization algorithm adopts a cascaded deep feature learning framework and realizes fine-grained similarity feature extraction through the following steps. First, multi-scale feature calculation is performed. Here, an improved Vision Transformer (ViT-L / 384) is used as the backbone network, and road scene feature domain adaptation training is performed on the basis of ImageNet-22K pre-training, so as to extract the hybrid feature descriptors containing global semantics (output of the GAP layer) and local details (attention map of the 12th layer) in the first image and the second image respectively, and calculate the class similarity feature values in the feature space, so as to obtain high-similarity image pairs. Through this high-precision feature calculation method, the accuracy and efficiency of image similarity processing are improved, and a set of image pairs with high similarity on the same link is also provided for subsequent tasks such as object recognition and status verification.

[0059] In some alternative implementation manners of this embodiment, the above method further includes: obtaining the road signs corresponding to the first image set; determining the images corresponding to the road signs from the historical image set to obtain a candidate image set; using a clustering algorithm to cluster the candidate image set to obtain a second image set, wherein the deviation between the vehicle heading angle corresponding to the second image in the second image set and the vehicle heading angle corresponding to the first image in the first image set is within a preset deviation range, and the offset between the coordinates of the second image and the coordinates of the first image is within a preset offset range.

[0060] In this implementation manner, the above-mentioned execution entity will first obtain the road signs corresponding to the images in the first image set. If the first image set includes consecutive images collected for multiple roads, the above-mentioned execution entity will first divide the new image set into multiple subsets according to the roads, each subset corresponding to the images of one road, and then obtain the road signs of the roads corresponding to each subset.

[0061] Then, the above-mentioned execution entity will obtain the images corresponding to the road signs from the historical image set (historically collected images). If there are multiple road sign links, the images corresponding to each road sign will be obtained respectively, so as to obtain a candidate image set.

[0062] After that, considering that some roads are two-way roads but are named with the same link, it is necessary to further determine the direction of the link. Here, the six-degree-of-freedom pose estimation of the first image can be combined with the SLAM (simultaneous localization and mapping) technology to determine the vehicle heading angle corresponding to the first image. Then, the spherical distance between the first image and the candidate image is calculated using the Haversine formula (a formula for calculating the distance between two longitudes and latitudes), thereby establishing a spatio-temporal correlation matrix containing the distance. Then, an improved DBSCAN (Density-Based Spatial Clustering of Applications with Noise) clustering algorithm is used, with the vehicle heading angle deviation (threshold ±15°) and coordinate offset (threshold ≤10 meters) as density clustering parameters, to generate a set of candidate images that are spatio-temporally adjacent, that is, the second image set.

[0063] Thus, the historical image set (the second image set) corresponding to the first image set is accurately determined based on spatial correlation.

[0064] Step 302, use the local feature matching algorithm to determine the pixel points in the second image that match the pixel points in the first image in the same image pair, and obtain similar pixel point pairs.

[0065] In this embodiment, for each image pair in at least one image pair (high similarity image pair), the above-mentioned execution entity will use a local feature matching algorithm (such as the LightGlue algorithm) to identify the combination of similar pixel points in the first image and the second image and the similarity score of each similar pixel point, thereby determining similar pixel point pairs.

[0066] Step 303, determine the detection frame corresponding to the candidate object in the first image and the similar pixel point pairs included in the detection frame.

[0067] In this embodiment, the above-mentioned execution entity will identify the first image and the second image in each image pair respectively. Specifically, the objects in the first image and the second image can be identified and located. Here, taking the object as a signboard for illustration, that is, the above-mentioned execution entity will first identify and locate the signboards in the first image and the second image to determine the positions of the signboards in the first image and the second image, and mark the signboards in the image, thereby obtaining the detection frame corresponding to the signboard. Then, the above-mentioned execution entity also constructs a feature mapping engine based on the detection frame and feature similarity points (or called feature matching points), that is, maps the detection frame coordinates to the feature matching space to determine the similar pixel point pairs included in the detection frame.

[0068] Step 304: In response to determining that the density of similar pixel pairs in the detection frame is greater than a preset density threshold, determine the candidate object as the target object.

[0069] In this embodiment, the above-mentioned execution entity calculates the density of similar pixel pairs. If it is determined that the density of similar pixel pairs in the detection frame is greater than a preset density threshold (for example, 15), then the candidate object is determined as the target object. Here, the candidate object is the object in the image pair, and the target object is the candidate object with a certain similarity in the image pair, that is, the candidate object that meets the similarity requirement in the first image and the second image is determined as the target object. Thus, the spatial density is combined to determine the target object with a certain similarity, improving the accuracy of object recognition.

[0070] In some alternative implementation manners of this embodiment, step 304 includes: calculating the spatial distribution variance of similar pixel pairs in the first image; in response to determining that the density is greater than the preset density threshold and the spatial distribution variance is less than the preset variance threshold, determine the candidate object as the target object.

[0071] In this implementation manner, the above-mentioned execution entity also calculates the spatial distribution variance of similar pixel pairs in the first image, and when the density is greater than the preset density threshold, further determines whether the spatial distribution variance is less than the preset variance threshold. When the density is greater than the preset density threshold and the spatial distribution variance is less than the preset variance threshold, the candidate object is determined as the target object. Thus, the distribution variance is combined on the basis of the spatial density to determine the target object, further improving the accuracy of object recognition.

[0072] It should be noted that if the density is less than the minimum density threshold, the target object can be determined as a low-confidence failure object. Here, the minimum density threshold is less than the preset density threshold mentioned above. For example, the preset density threshold can take a value of 15, while the minimum density threshold can take a value of 5.

[0073] Step 305: Determine the object category of the target object in the image pair according to the feature similarity between the target object in the first image and the target object in the second image in the image pair.

[0074] Step 306: Statistically count the object categories of the target object in at least one image pair, and determine the status information of the target object according to the statistical results.

[0075] Steps 305-306 are basically the same as steps 203-204 in the foregoing embodiment. The specific implementation manners can refer to the description of steps 203-204 above and will not be elaborated here.

[0076] From Figure 3 it can be seen that compared with Figure 2Compared with the corresponding embodiments, in this embodiment, the method for determining information validity highlights the steps of determining the second image set and the target object. It accurately determines the historical image set (the second image set) corresponding to the first image set based on spatial correlation, and determines the target object in the image pair by combining the distribution variance on the basis of spatial density, thereby improving the accuracy of object recognition.

[0077] Continue to refer to Figure 4 , Figure 4 which shows a flow 400 of another embodiment of the method for determining information validity according to the present disclosure. The method for determining information validity includes the following steps:

[0078] Step 401, pair the images in the first image set and the second image set according to the similarity between image features to obtain at least one image pair.

[0079] Step 402, determine the target object in the image pair according to the matching result of the image pixel points in the image pair.

[0080] Steps 401-402 are basically the same as steps 201-202 of the foregoing embodiments. The specific implementation manner can refer to the description of steps 201-202 above and will not be elaborated here.

[0081] Step 403, calculate the cosine similarity between the features of the target object in the first image and the features of the target object in the second image.

[0082] In this embodiment, the execution subject of the method for determining information validity (such as Figure 1 the server 105 shown) constructs a 256-dimensional deep feature encoder to perform feature encoding on the target objects in the first image and the second image in the image pair, calculates the cosine similarity between the target objects in the first image and the second image according to the feature encoding, and designs a dynamic threshold decision mechanism, so as to determine the object category of the target object according to the dynamic threshold decision mechanism.

[0083] Step 404, in response to determining that the cosine similarity is greater than the first similarity threshold, determine that the object category of the target object in the image pair is a high-confidence valid object.

[0084] In this embodiment, if it is determined that the cosine similarity is greater than the first similarity threshold, the above-mentioned execution entity will determine that the object category of the target object in the image pair is a high-confidence valid object and put the target object into the high-confidence valid set. The first similarity threshold can be set to 0.95, that is, if the cosine similarity is greater than 0.95 or greater than or equal to 0.95, it is determined that the object category of the target object in the image pair is a high-confidence valid object. Thus, the object category of the target object can be accurately determined according to the cosine similarity of the target objects in the two images of the image pair. When the cosine similarity is greater than the threshold, the target object is determined to be an object with high confidence and is determined to be a valid object, thereby improving the accuracy of the object category information.

[0085] Step 405: In response to determining that the cosine similarity is less than the second similarity threshold, determine that the object category of the target object in the image pair is a low-confidence invalid object.

[0086] In this embodiment, if it is determined that the cosine similarity is less than the second similarity threshold, the above-mentioned execution entity will determine that the object category of the target object in the image pair is a low-confidence invalid object, where the second similarity threshold is less than the first similarity threshold.

[0087] If it is determined that the cosine similarity is less than the second similarity threshold, the above-mentioned execution entity will determine that the object category of the target object in the image pair is a low-confidence invalid object and put the target object into the low-confidence set. The second similarity threshold can be set to 0.85, that is, if the cosine similarity is less than 0.85, it is determined that the object category of the target object in the image pair is a low-confidence invalid object. Thus, when the cosine similarity is less than the threshold, the target object is determined to be an object with low confidence and is determined to be an invalid object, thereby improving the accuracy of the object category information.

[0088] Step 406: In response to determining that the cosine similarity is greater than the second similarity threshold and less than the first similarity threshold, calculate the similarities between the target object in the first image and the same object in the pre-constructed database, and between the target object in the second image and the same object in the pre-constructed database, to obtain a first similarity and a second similarity.

[0089] In this embodiment, if it is determined that the cosine similarity is between the first similarity threshold and the second similarity threshold, that is, the cosine similarity is greater than the second similarity threshold and less than the first similarity threshold, at this time, the object category of the target object cannot be determined, and the above-mentioned execution entity will further perform a relevance verification to determine the object category of the target object according to the verification result.

[0090] Specifically, the above-mentioned execution entity constructs a spatial image feature map, and calculates the similarity between the target object in the first image and the target object in the second image and the same object in the pre-constructed database respectively, so as to obtain the first similarity and the second similarity. For example, when the object is a signboard, the above-mentioned execution entity constructs a spatial image feature map at the signboard level and calculates the similarity between the target signboard and the signboards in the database.

[0091] Step 407, in response to determining that the first similarity and / or the second similarity does not meet the preset similarity requirement, determine that the object category of the target object in the image pair is a low-confidence failure object.

[0092] In this embodiment, if it is determined that one or both of the first similarity and the second similarity do not meet the preset similarity requirement, that is, the target object in the first image and the target object in the second image are not associated with the same object in the database. At this time, the above-mentioned execution entity will determine that the object category of the target object in the image pair is a low-confidence failure object and put the target object into the low-confidence set. Thus, when the object category of the target object cannot be accurately determined according to the cosine similarity, the relevance verification of the target object is further carried out. When it does not meet the verification requirement, it is determined that the object category of the target object is a low-confidence failure object, thereby improving the accuracy of the object category information.

[0093] Step 408, in response to determining that both the first similarity and the second similarity meet the preset similarity requirement, determine that the object category of the target object in the image pair is a high-confidence valid object.

[0094] In this embodiment, if it is determined that both the first similarity and the second similarity meet the preset similarity requirement, that is, the target object in the first image and the target object in the second image are both associated with the same object in the database. At this time, the above-mentioned execution entity will determine that the object category of the target object in the image pair is a high-confidence valid object and put the target object into the high-confidence valid set. Thus, when the object category of the target object cannot be accurately determined according to the cosine similarity, the relevance verification of the target object is further carried out. When it meets the verification requirement, it is determined that the object category of the target object is a high-confidence valid object, thereby improving the accuracy of the object category information.

[0095] Step 409, filter and verify the low-confidence failure objects, and re-determine the object category of the target object according to the verification result.

[0096] In this embodiment, the above-mentioned execution entity will also filter and verify the low-confidence failure objects in the low-confidence set to re-determine the object category of the target object, that is, filter out high-confidence valid objects or high-confidence failure objects from the low-confidence failure objects. That is to say, the process of verifying the high-confidence objects has been completed in the previous steps, and here the low-confidence failure objects will be filtered again to exclude misjudgment situations.

[0097] In some optional implementation manners of this embodiment, the above method further includes: determining a first position box of the low-confidence failure object in the first image; projecting the low-confidence failure object in the first image onto the second image, and determining a second position box of the low-confidence failure object in the second image according to the projection result.

[0098] In this implementation manner, the above-mentioned execution entity will first detect and locate the low-confidence failure objects in the first image, so as to determine the positions of the low-confidence objects in the first image and obtain the corresponding first position box. Then, the above-mentioned execution entity will perform projection matrix optimization modeling on the low-confidence failure objects, and iteratively solve the optimal projection parameters through the LM (Levenberg-Marquardt) algorithm. Specifically, the target object in the first image is projected into the second image through the projection matrix to obtain a projection result; and the second position box of the low-confidence failure object in the second image is determined according to the projection result, so as to identify the object position box in the real world at the same position.

[0099] And step 409 further includes: calculating the size ratio of the first position box and the second position box; in response to determining that the size ratio is within a preset ratio range, determining the low-confidence failure object as an object to be verified; performing relevance verification on the object to be verified, and determining the object category of the target object as a high-confidence valid object or a high-confidence failure object according to the verification result.

[0100] Here, the above-mentioned execution entity will perform size anomaly detection on the low-confidence failure objects, establish a size change rate strategy model, and determine the size ratio of the first position box and the second position box through this model. If the size ratio is within the preset ratio range, it will be filtered out, and the filtered low-confidence failure objects will be determined as objects to be verified for relevance verification of the objects to be verified.

[0101] After that, the above-mentioned execution entity will perform relevance verification on the filtered low-confidence failure objects (i.e., objects to be verified) again, and determine whether the object category of the target object is a high-confidence valid object or a high-confidence failure object according to the verification result.

[0102] By filtering out low-confidence failure objects with large variations in area size, the situation of misjudgment caused by large variations in area size is avoided, and the filtered low-confidence failure objects are verified for relevance again, which can further improve the accuracy of object category information.

[0103] In some alternative implementation manners of this embodiment, the above method further includes: determining a first position box of the low-confidence failure object in the first image. That is, the above execution entity first detects and locates the low-confidence failure object in the first image, so as to determine the position of the low-confidence object in the first image and obtain the corresponding first position box.

[0104] And step 409 further includes: calculating the proportion of the occluded area according to the mapping relationship between the first position box and each segmentation area of the first image; in response to determining that the proportion of the occluded area is within a preset proportion range, determining the low-confidence failure object as an object to be verified; performing relevance verification on the object to be verified, and determining the object category of the target object as a high-confidence valid object or a high-confidence failure object according to the verification result.

[0105] In this implementation manner, the above execution entity also uses a deep learning algorithm based on InternImage to perform fine-grained segmentation on the image. There are 40 segmentation categories, including buildings, vehicles, trees, railings, gas stations, etc., which are mainly used to judge the occlusion situation. This step ensures the accurate recognition of objects in a complex environment and provides strong support for filtering out objects that change due to occlusion in the subsequent process. Then, the above execution entity performs occlusion analysis on the low-confidence failure object, that is, calculates the proportion of the occluded area according to the mapping relationship between the first position box and each segmentation area (obtained through the previous image segmentation step) of the first image. If the proportion of the occluded area is within the preset proportion range, it is filtered out, and the filtered low-confidence failure object is determined as an object to be verified for relevance verification of the object to be verified.

[0106] After that, the filtered low-confidence failure objects (i.e., objects to be verified) are verified for relevance again, and the object category of the target object is determined as a high-confidence valid object or a high-confidence failure object according to the verification result.

[0107] By filtering out low-confidence failure objects with occlusion situations, the situation of misjudgment caused by occlusion is avoided, and the filtered low-confidence failure objects are verified for relevance again, further improving the accuracy of object category information.

[0108] Of course, in this embodiment, it is also possible to determine whether a low-confidence object is an object to be verified according to the size ratio and the occlusion area ratio, that is, to determine a low-confidence object with a size ratio within a preset ratio range and an occlusion area ratio within a preset ratio range as an object to be verified. The size ratio and the occlusion area ratio can be calculated according to the foregoing content and will not be elaborated here. In this way, low-confidence failure objects that simultaneously have large area size changes and occlusion situations are filtered out, so that low-confidence objects determined due to misjudgment can be more accurately filtered, and the filtered low-confidence failure objects are verified for relevance again, which can further improve the accuracy of object category information.

[0109] In some alternative implementation manners of this embodiment, for relevance verification of an object to be verified, and determining the object category of the target object as a high-confidence valid object or a high-confidence failure object according to the verification result, includes: calculating the similarity between the object to be verified in the first image and the object to be verified in the second image and the same object in the database respectively, to obtain a third similarity and a fourth similarity; in response to determining that both the third similarity and the fourth similarity meet the preset similarity requirements, determining the object category of the target object as a high-confidence valid object.

[0110] In this implementation manner, based on the established topological graph, recalculate the similarity between the object to be verified in the first image and the object to be verified in the second image and the same object in the database, to obtain a third similarity and a fourth similarity. If it is determined that both the third similarity and the fourth similarity meet the preset similarity requirements, that is, the object to be verified in the first image and the object to be verified in the second image are both associated with the same object in the database, then determine that this object to be verified is a high-confidence valid object and put it into the high-confidence valid set.

[0111] Thus, relevance verification is performed on low-confidence failure objects. When it meets the verification requirements, determine the object category of the target object as a high-confidence valid object, thereby improving the accuracy of object category information.

[0112] In some alternative implementation manners of this embodiment, the above method further includes: in response to determining that the third similarity and / or the fourth similarity does not meet the preset similarity requirements, determining the object category of the target object as a high-confidence failure object.

[0113] In this implementation manner, if the third similarity and / or the fourth similarity does not meet the preset similarity requirements, that is, the object to be verified in the first image and the object to be verified in the second image are not associated with the same object in the database, then determine the object category of this target object as a high-confidence failure object and put it into the high-confidence failure set.

[0114] Therefore, the relevance verification is performed on the object to be verified. When it does not meet the verification requirements, the category of the target object is determined as a high-confidence failure object, thereby improving the accuracy of the object category information.

[0115] Step 410: Count the object categories of the target object in at least one pair of images, and determine the status information of the target object according to the statistical results.

[0116] Step 410 is basically the same as step 204 of the foregoing embodiment. The specific implementation manner can refer to the description of step 204 above and will not be elaborated here.

[0117] From Figure 4 it can be seen that compared with the corresponding embodiment of Figure 3 , in the method for determining the information validity in this embodiment, this method highlights the step of determining the object category of the target object according to multi-level verification, and the misjudgment filtering mechanism constructed. The low-confidence failure objects are filtered, and the low-confidence failure objects with large area size changes and occlusion conditions are filtered out, and the relevance verification is performed again on the filtered low-confidence failure objects, thereby avoiding misjudgment caused by large area size changes and occlusion conditions, and further improving the accuracy of the object category information. This method uses the verification results of the image pairs for cross-verification to ensure the accuracy of the final obtained object category (high-confidence valid object, high-confidence failure object, and low-confidence failure object) results.

[0118] Continue to refer to Figure 5 , Figure 5 shows Figure 4 a process 500 of the step of determining the status information of the target object in

[0119] Step 501: In response to determining that the number of times the object category of the target object is determined as a high-confidence valid object according to the statistical results is greater than a preset valid number threshold, determine the status information of the target object as a valid state.

[0120] If the statistical results show that the number of times the target object is determined as a high-confidence valid object is greater than the valid number threshold, the above-mentioned execution entity will determine the final state of the target object as valid. The valid number threshold here can be 3 times, that is, if the number of times the target object is determined as a high-confidence valid object is greater than 3 times, the final state of the target object is determined as valid.

[0121] Step 502: In response to determining that the number of times the object category of the target object is determined as a high-confidence failure object according to the statistical results is greater than a preset failure number threshold, determine the status information of the target object as a failure state.

[0122] If the statistical result shows that the number of times the target object is determined to be a high-confidence failure object is greater than the failure times threshold, the above-mentioned execution entity will determine the final state of the target object as failed. The failure times threshold here can be 2 times, that is, if the number of times the target object is determined to be a high-confidence failure object is greater than 2 times, then the final state of the target object is determined to be failed.

[0123] Step 503, in response to determining that the number of times the object category of the target object is determined to be a low-confidence failure object according to the statistical result is less than the failure times threshold, determine the status information of the target object as an unclear state.

[0124] If the statistical result shows that the number of times the target object is determined to be a low-confidence failure object is less than the failure times threshold, the above-mentioned execution entity will determine the final state of the target object as an unclear state. That is, if the number of times the target object is determined to be a low-confidence failure object is less than 2 times, then the final state of the target object is determined to be an unclear state.

[0125] Thus, after completing the calculation of a single image pair, the object category of the target object in all image pairs is statistically counted, and all calculation results are clustered in terms of time and feature dimensions through object features to form multiple clustering sets, and the final status information of the target object is determined according to the number after clustering, thereby ensuring the accuracy of the status information of the target object.

[0126] Continue to refer to Figure 6 , Figure 6 shows the flow 600 of another embodiment of the method for determining information validity according to the present disclosure. The method for determining information validity includes the following steps:

[0127] Step 601, pair the images in the first image set and the second image set according to the similarity between the image features to obtain at least one image pair.

[0128] Step 602, determine the target object in the image pair according to the matching result of the image pixels in the image pair.

[0129] Step 603, determine the object category of the target object in the image pair according to the feature similarity between the target object in the first image and the target object in the second image in the image pair.

[0130] Step 604, statistically count the object categories of the target object in at least one image pair, and determine the status information of the target object according to the statistical result.

[0131] Steps 601-604 are basically the same as steps 201-204 of the foregoing embodiment, and the specific implementation manner can refer to the description of steps 201-204 above, and will not be elaborated here.

[0132] Step 605: In response to determining that the status information of the target object is in a valid state, update the information of the point of interest corresponding to the target object in the map database.

[0133] In this embodiment, if it is determined that the status information of the target object is in a valid state, the execution entity of the method for determining information validity (such as Figure 1 the server 105 shown) then confirms that the target object still exists. At this time, the information of the target object will be used to update the information of the point of interest corresponding to the target object in the map database. Natural landscapes, commercial places, public facilities, etc. in the real world are all points of interest in the map, such as stores, bus stops, mailboxes, etc. That is, the target object in this embodiment has a corresponding point of interest in the map. The above execution entity will use the information of the target object for database update and establish a feature incremental update mechanism to dynamically optimize the features in the feature library.

[0134] Step 606: In response to determining that the status information of the target object is in an invalid state, delete the information of the point of interest corresponding to the target object in the map database.

[0135] In this embodiment, if it is determined that the status information of the target object is in an invalid state, the above execution entity determines that the target object has failed (such as being demolished) or has been replaced. At this time, the information of the point of interest corresponding to the target object in the map database is taken offline, that is, the information of the point of interest corresponding to the target object in the map database is deleted.

[0136] Step 607: In response to determining that the status information of the target object is in an unclear state, generate a secondary collection task to re-determine the status information of the target object.

[0137] In this embodiment, if it is determined that the status information of the target object is in an unclear state, the above execution entity will construct an active learning mechanism, automatically generate a secondary collection task, and develop an uncertainty quantification model to preferentially collect high-information-entropy regions to re-determine the status information of the target object.

[0138] From Figure 6 it can be seen that compared with the corresponding embodiment of Figure 2 , the method for determining information validity in this embodiment highlights processing the information of the point of interest corresponding to the target object in the map database according to the status of the target object, thereby timely maintaining the point of interest information in the map database and improving the accuracy of the information in the map database.

[0139] Further referring to Figure 7 , Figure 7 shows an application process of the method for determining information validity according to the present disclosure, including the following steps:

[0140] S701: Collect the road image data captured by vehicle equipment, and construct new and old image datasets based on spatial correlation (link).

[0141] This step is completed by the data acquisition and construction module, which adopts multi-source heterogeneous data fusion technology. Through multi-modal sensors carried by the vehicle (including high-precision GPS, IMU, 360° panoramic cameras), it can collect road images and spatial coordinate information in real time. Based on the hierarchical indexing structure of geographical spatial road signs (link) (constructed by using R-tree spatial database), through the spatio-temporal joint coding strategy, old images belonging to the same road as the newly collected images can be recalled from a large number of historical images, and the recall accuracy is increased to 98.7%.

[0142] S702: Perform matching of new and old images through visual positioning algorithms to generate a set of image pairs with high similarity.

[0143] This step is completed by the spatio-temporal correlation data recall module, which adopts visual positioning algorithms. This algorithm is based on a deep learning network and can encode images into a series of feature values. By comparing the feature value sequences of the new and old images, the similarity of the image pairs can be evaluated, and thus a set of image pairs with high similarity can be obtained. The feature values of the image pairs with high similarity have small differences, indicating that the two images are highly consistent in visual content. In addition, this algorithm can effectively filter out those image pairs with low similarity, thus ensuring that the output set of image pairs is highly consistent visually.

[0144] S703: Parallelly execute for a single group of image pairs: sign target detection, scene semantic segmentation, and feature point matching.

[0145] This step is completed by the multi-modal feature fusion module. After obtaining multiple sets of image pair sets, this module calculates for each single pair of new and old images respectively, including sign detection, image segmentation, and image pair feature matching, and establishes an information interaction channel across subsystems to achieve three-dimensional feature fusion of detection boxes - segmentation masks - matching points.

[0146] S704: Establish a mapping relationship based on the detection box coordinates and feature matching points to generate a candidate set of suspected similar / failed signs.

[0147] At this stage, the system constructs a feature mapping engine based on the detection box coordinates and feature matching points. By mapping the detection box coordinates to the feature matching space, it can effectively identify the candidate sets of suspected similar signs and suspected failed signs. Specifically, when the matching point density ≥ 15 and the spatial distribution variance < 0.2, it is determined as the candidate set of suspected similar signs; if the matching point density < 5, it is determined as the candidate set of suspected failed signs. Thus, by combining spatial density and distribution variance, the accuracy of recognition is improved.

[0148] S705: Encode the signs' features for the suspected similar candidate set, and filter through the feature similarity network to form a highly confident similar set. The dissimilar part is regarded as the suspected invalid signs.

[0149] S706: Conduct database association verification on the highly confident similar set by combining spatial topological relationships. Transfer those that match the same sign into the suspected invalid set. If the association is consistent, output it as the similar sign set.

[0150] S707: Conduct projection matrix calculation and feature point constraint verification on the suspected invalid set, and output the low-confidence invalid signs.

[0151] S705 - S707 perform a multi-level verification decision tree to obtain a highly confident similar set and a suspected invalid candidate set. The verification strategies include feature-level verification, association-level verification, and geometric-level verification.

[0152] Feature-level verification: Construct a 256-dimensional deep feature encoder to encode the signs' features for the suspected similar candidate set, and design a dynamic threshold decision mechanism. If the cosine similarity ≥ 0.95, it enters the highly confident verification set. If it is between 0.85 - 0.95, it triggers the association-level verification. If it is less than 0.85, it is marked as the suspected invalid set.

[0153] Association-level verification: Construct a topological map of the spatial image features at the sign level, and calculate the similarity between the candidate sign and the signs in the database. If the new and old signs are both associated with the same sign, it is considered that this sign belongs to the highly confident verification set; otherwise, it is marked as the suspected invalid set.

[0154] Geometric-level verification: Optimize and model the projection matrix for the suspected invalid set, and iteratively solve the optimal projection parameters through the LM algorithm. Then, project the signs in the new image into the old image through the projection matrix to identify the real-world sign images at the same position.

[0155] S708: Filter the low-confidence invalid set, exclude misjudgments caused by large sign size differences, small area, or occlusion, etc., and further conduct association verification using spatial topological relationships to finally obtain a highly confident invalid sign set.

[0156] Here, the suspected invalid set is regarded as the low-confidence invalid set, and the low-confidence invalid set is filtered. Specifically:

[0157] Abnormal size detection: Establish a strategy model for the sign size change rate, filter out signs with a too large aspect ratio difference between the widths and heights in two images of the same sign, and put them into the set to be verified.

[0158] Occluded sign analysis: Based on the mapping relationship between the sign frame and the segmentation area obtained in the S703 stage, obtain the proportion of the occluded area of the sign. If the occlusion is severe, filter it and put it into the set to be verified.

[0159] Associated anomaly detection: Based on the topological map established in the previous stage, recalculate the similarity between the signs in the set to be verified and the signs in the database. If the old and new signs are both associated with the same sign, then this sign belongs to the high-confidence verification set; otherwise, it is classified into the high-confidence failure set.

[0160] After the calculation for a single pair of images is completed, repeat steps S703 - S708. After all image pairs are calculated, transfer to S709.

[0161] S709: After all image pairs are calculated, perform clustering based on the sign features to obtain a clustering set of verified or failed signs within multiple consecutive images, and distinguish different confidence levels based on the number after clustering.

[0162] Steps S703 - S709 are all completed by the intelligent decision verification module. After the calculation for a single pair of images is completed, the system uses the intelligent clustering engine to develop a spatio-temporal continuity clustering strategy. All calculation results are clustered based on the sign features in the time and feature dimensions to form a clustering set of verified or failed signs within multiple consecutive images, and different confidence levels are set according to the number after clustering:

[0163] High-confidence verification set: Signs verified by image pairs more than 3 times;

[0164] High-confidence failure set: Signs verified as failed by image pairs 2 times or more;

[0165] Low-confidence failure set: Signs verified as failed by image pairs less than 2 times.

[0166] S710: According to the processing results, divide the signs into three sets, namely: high-confidence verification set, high-confidence failure set, and low-confidence failure set.

[0167] This step is completed by the multi-modal feature fusion module, which divides the signs into three sets according to the processing results. Different sets are transferred to different stages within the system. Specifically:

[0168] High-confidence verification set: Confirm that the sign still exists, and use the information of this sign to update the database;

[0169] High-confidence failure set: The sign is confirmed to be failed or replaced, and the relevant data of the sign is taken offline;

[0170] Low-confidence failure set: For signs with unclear status, further verification is required. An active learning mechanism can be constructed to automatically generate a secondary acquisition task.

[0171] For further reference Figure 8 , as an implementation of the methods shown in the above figures, the present disclosure provides an embodiment of a device for determining information validity. This device embodiment is related to Figure 2The method embodiments shown correspond to this, and this device can be specifically applied to various electronic devices.

[0172] As Figure 8 As shown, the information validity determination device 800 in this embodiment includes: an image pair determination module 801, a target object determination module 802, an object category determination module 803, and a status information determination module 804. Among them, the image pair determination module 801 is configured to pair the images in the first image set and the second image set according to the similarity between image features, to obtain at least one image pair, where the acquisition time of the second image set is earlier than that of the first image set; the target object determination module 802 is configured to determine the target object in the image pair according to the matching result of the image pixel points in the image pair; the object category determination module 803 is configured to determine the object category of the target object in the image pair according to the feature similarity between the target object in the first image and the target object in the second image in the image pair; the status information determination module 804 is configured to count the object categories of the target object in at least one image pair, and determine the status information of the target object according to the statistical result, where the status information includes: valid status, invalid status, and unclear status.

[0173] In this embodiment, in the information validity determination device 800: the specific processing of the image pair determination module 801, the target object determination module 802, the object category determination module 803, and the status information determination module 804 and the technical effects brought by them can respectively refer to Figure 2 the relevant descriptions of steps 201-204 in the corresponding embodiments, which will not be elaborated here.

[0174] In some optional implementation manners of this embodiment, the above information validity determination device 800 further includes: an acquisition module, configured to acquire the road signs corresponding to the first image set; a candidate module, configured to determine the images corresponding to the road signs from the historical image set to obtain a candidate image set; a clustering module, configured to cluster the candidate image set by using a clustering algorithm to obtain a second image set, where the deviation between the vehicle heading angle corresponding to the second image in the second image set and the vehicle heading angle corresponding to the first image in the first image set is within a preset deviation range, and the offset between the coordinates of the second image and the coordinates of the first image is within a preset offset range.

[0175] In some alternative implementation manners of this embodiment, the target object determination module 802 includes: a matching sub-module configured to use a local feature matching algorithm to determine the pixel points in the second image that match the pixel points in the first image in the same image pair, and obtain similar pixel point pairs; a first determination sub-module configured to determine the detection frame corresponding to the candidate object in the first image and the similar pixel point pairs included in the detection frame; a second determination sub-module configured to, in response to determining that the density of the similar pixel point pairs in the detection frame is greater than a preset density threshold, determine the candidate object as the target object.

[0176] In some alternative implementation manners of this embodiment, the second determination sub-module is further configured to: calculate the spatial distribution variance of the similar pixel point pairs in the first image; in response to determining that the density is greater than the preset density threshold and the spatial distribution variance is less than the preset variance threshold, determine the candidate object as the target object.

[0177] In some alternative implementation manners of this embodiment, the object category determination module 803 is further configured to: calculate the cosine similarity between the features of the target object in the first image and the features of the target object in the second image; in response to determining that the cosine similarity is greater than the first similarity threshold, determine that the object category of the target object in the image pair is a high-confidence valid object.

[0178] In some alternative implementation manners of this embodiment, the above-mentioned information validity determination device 800 further includes: a first determination module configured to, in response to determining that the cosine similarity is less than the second similarity threshold, determine that the object category of the target object in the image pair is a low-confidence invalid object, where the second similarity threshold is less than the first similarity threshold.

[0179] In some alternative implementation manners of this embodiment, the above-mentioned information validity determination device 800 further includes: a calculation module configured to, in response to determining that the cosine similarity is greater than the second similarity threshold and less than the first similarity threshold, calculate the similarities between the target object in the first image and the target object in the second image and the same object in the pre-constructed database respectively, to obtain a first similarity and a second similarity, where the second similarity threshold is less than the first similarity threshold; a second determination module configured to, in response to determining that both the first similarity and the second similarity meet the preset similarity requirements, determine that the object category of the target object in the image pair is a high-confidence valid object.

[0180] In some alternative implementation manners of this embodiment, the above-mentioned information validity determination device 800 further includes: a third determination module configured to, in response to determining that the first similarity and / or the second similarity do not meet the preset similarity requirements, determine that the object category of the target object in the image pair is a low-confidence invalid object.

[0181] In some alternative implementation manners of this embodiment, the above-mentioned determining device 800 for information validity further includes: a filtering module, configured to filter and verify low-confidence failure objects, and re-determine the object category of the target object according to the verification result.

[0182] In some alternative implementation manners of this embodiment, the above-mentioned determining device 800 for information validity further includes: a first position box determining module, configured to determine a first position box of a low-confidence failure object in a first image; a projection module, configured to project the low-confidence failure object in the first image onto a second image, and determine a second position box of the low-confidence failure object in the second image according to the projection result; and the filtering module includes: a first calculation sub-module, configured to calculate a size ratio of the first position box and the second position box; a first verification sub-module, configured to determine the low-confidence failure object as an object to be verified in response to determining that the size ratio is within a preset ratio range; a second verification sub-module, configured to perform a relevance verification on the object to be verified, and determine the object category of the target object as a high-confidence valid object or a high-confidence failure object according to the verification result.

[0183] In some alternative implementation manners of this embodiment, the above-mentioned determining device 800 for information validity further includes: a second position box determining module, configured to determine a first position box of a low-confidence failure object in a first image; and the filtering module includes: a second calculation sub-module, configured to calculate an occlusion area ratio according to a mapping relationship between the first position box and each segmentation area of the first image; a second verification sub-module, configured to determine the low-confidence failure object as an object to be verified in response to determining that the occlusion area ratio is within a preset ratio range; a third verification sub-module, configured to perform a relevance verification on the object to be verified, and determine the object category of the target object as a high-confidence valid object or a high-confidence failure object according to the verification result.

[0184] In some alternative implementation manners of this embodiment, the second verification sub-module or the third verification sub-module is further configured to: calculate a third similarity and a fourth similarity between the object to be verified in the first image and the object to be verified in the second image and the same object in the database respectively; and determine the object category of the target object as a high-confidence valid object in response to determining that both the third similarity and the fourth similarity meet the preset similarity requirements.

[0185] In some alternative implementation manners of this embodiment, the above-mentioned determining device 800 for information validity further includes: a fourth verification sub-module, configured to determine the object category of the target object as a high-confidence failure object in response to determining that the third similarity and / or the fourth similarity does not meet the preset similarity requirements.

[0186] In some alternative implementation manners of this embodiment, the status information determination module is further configured to: in response to determining that the number of times the object category of the target object is determined to be a high-confidence valid object according to the statistical result is greater than a preset valid number threshold, determine the status information of the target object as a valid status; in response to determining that the number of times the object category of the target object is determined to be a high-confidence invalid object according to the statistical result is greater than a preset invalid number threshold, determine the status information of the target object as an invalid status; in response to determining that the number of times the object category of the target object is determined to be a low-confidence invalid object according to the statistical result is less than the invalid number threshold, determine the status information of the target object as an unclear status.

[0187] In some alternative implementation manners of this embodiment, the above-mentioned determination device 800 for information validity further includes: an update module, configured to update the information of the point of interest corresponding to the target object in the map database in response to determining that the status information of the target object is a valid status; a deletion module, configured to delete the information of the point of interest corresponding to the target object in the map database in response to determining that the status information of the target object is an invalid status; a collection module, configured to generate a secondary collection task in response to determining that the status information of the target object is an unclear status, so as to re-determine the status information of the target object.

[0188] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0189] Figure 9 FIG. shows a schematic block diagram of an exemplary electronic device 900 that can be used to implement the embodiments of the present disclosure. The electronic device is intended to represent various forms of digital computers, such as, a laptop computer, a desktop computer, a workbench, a personal digital assistant, a server, a blade server, a mainframe computer, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as, a personal digital processing, a cellular phone, a smart phone, a wearable device, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0190] As Figure 9 shown, the device 900 includes a computing unit 901, which can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 902 or the computer program loaded from the storage unit 908 into the random access memory (RAM) 903. In the RAM 903, various programs and data required for the operation of the device 900 can also be stored. The computing unit 901, the ROM 902, and the RAM 903 are connected to each other through a bus 904. The input / output (I / O) interface 905 is also connected to the bus 904.

[0191] Multiple components in device 900 are connected to I / O interface 905, including: input unit 906, such as a keyboard, mouse, etc.; output unit 907, such as various types of displays, speakers, etc.; storage unit 908, such as a disk, optical disc, etc.; and communication unit 909, such as a network card, modem, wireless communication transceiver, etc. Communication unit 909 allows device 900 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0192] Computing unit 901 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of computing unit 901 include but are not limited to a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Computing unit 901 executes the various methods and processes described above, such as the method for determining information validity. For example, in some embodiments, the method for determining information validity can be implemented as a computer software program that is tangibly contained in a machine-readable medium, such as storage unit 908. In some embodiments, part or all of the computer program can be loaded and / or installed onto device 900 via ROM 902 and / or communication unit 909. When the computer program is loaded into RAM 903 and executed by computing unit 901, one or more steps of the method for determining information validity described above can be executed. Alternatively, in other embodiments, computing unit 901 can be configured to execute the method for determining information validity in any other suitable manner (e.g., by means of firmware).

[0193] The various embodiments of the systems and techniques described above in this document can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-chip systems (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special or general-purpose programmable processor, and can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit the data and instructions to the storage system, the at least one input device, and the at least one output device.

[0194] The program code for implementing the methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general purpose computer, a special purpose computer, or other programmable data processing apparatus, such that the program codes, when executed by the processor or controller, cause the functions / operations specified in the flowchart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, as a stand-alone software package partially on the machine and partially on a remote machine, or entirely on the remote machine or server.

[0195] In the context of the present disclosure, a machine-readable medium may be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0196] In order to provide interaction with a user, the systems and techniques described herein may be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0197] The systems and techniques described herein can be implemented in a computing system including backend components (e.g., as a data server), or a computing system including middleware components (e.g., an application server), or a computing system including frontend components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described herein), or a computing system including any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected to each other by digital data communication in any form or medium (e.g., a communication network). Examples of communication networks include: local area network (LAN), wide area network (WAN), and the Internet.

[0198] A computer system can include a client and a server. The client and the server are generally remote from each other and typically interact through a communication network. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, a server of a distributed system, or a server incorporating blockchain.

[0199] It should be understood that the various forms of the processes shown above can be used, with steps reordered, added, or deleted. For example, the steps recited in this disclosure can be executed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this is not limited herein.

[0200] The above specific embodiments do not constitute a limitation on the protection scope of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure shall be included within the protection scope of this disclosure.

Claims

1. A method for determining information validity, comprising: Pairing the images in the first image set and the second image set according to the similarity between image features to obtain at least one image pair, wherein the acquisition time of the second image set is earlier than that of the first image set; Determining the target object in the image pair according to the matching result of the image pixel points in the image pair; Determining the object category of the target object in the image pair according to the feature similarity between the target object in the first image and the target object in the second image in the image pair; Counting the object categories of the target object in the at least one image pair, and determining the status information of the target object according to the statistical result, wherein the status information includes: valid status, invalid status, and unclear status.

2. The method according to claim 1, further comprising: Obtaining the road signs corresponding to the first image set; Determining the images corresponding to the road signs from the historical image set to obtain a candidate image set; Clustering the candidate image set by using a clustering algorithm to obtain the second image set, wherein the deviation between the vehicle heading angle corresponding to the second image in the second image set and the vehicle heading angle corresponding to the first image in the first image set is within a preset deviation range, and the offset between the coordinates of the second image and the coordinates of the first image is within a preset offset range.

3. The method according to claim 1, wherein, The determining the target object in the image pair according to the matching result of the image pixel points in the image pair includes: Using a local feature matching algorithm to determine the pixel points in the second image that match the pixel points in the first image in the same image pair to obtain similar pixel point pairs; Determining the detection frame corresponding to the candidate object in the first image and the similar pixel point pairs included in the detection frame; In response to determining that the density of the similar pixel point pairs in the detection frame is greater than a preset density threshold, determining the candidate object as the target object.

4. The method according to claim 3, wherein The in response to determining that the density of the similar pixel point pairs in the detection frame is greater than a preset density threshold, determining the candidate object as the target object includes: Calculating the spatial distribution variance of the similar pixel point pairs in the first image; In response to determining that the density is greater than the preset density threshold and the spatial distribution variance is less than a preset variance threshold, determining the candidate object as the target object.

5. The method according to claim 1, wherein The determining the object category of the target object in the image pair according to the feature similarity between the target object in the first image and the target object in the second image in the image pair includes: Calculating the cosine similarity between the features of the target object in the first image and the features of the target object in the second image; In response to determining that the cosine similarity is greater than a first similarity threshold, determining the object category of the target object in the image pair as a high-confidence valid object.

6. The method according to claim 5, further comprising: In response to determining that the cosine similarity is less than a second similarity threshold, it is determined that the object category of the target object in the image pair is a low-confidence failure object, where the second similarity threshold is less than the first similarity threshold.

7. The method according to claim 6, further comprising: In response to determining that the cosine similarity is greater than the second similarity threshold and less than the first similarity threshold, calculate the similarity between the target object in the first image and the same object in a pre-constructed database, and the similarity between the target object in the second image and the same object in the pre-constructed database, to obtain a first similarity and a second similarity; In response to determining that both the first similarity and the second similarity meet a preset similarity requirement, determine that the object category of the target object in the image pair is the high-confidence valid object.

8. The method according to claim 7, further comprising: In response to determining that the first similarity and / or the second similarity do not meet the preset similarity requirement, determine that the object category of the target object in the image pair is the low-confidence failure object.

9. The method according to claim 8, further comprising: Filter and verify the low-confidence failure object, and re-determine the object category of the target object according to the verification result.

10. The method according to claim 9, further comprising: Determine a first position box of the low-confidence failure object in the first image; Project the low-confidence failure object in the first image onto the second image, and determine a second position box of the low-confidence failure object in the second image according to the projection result; and The filtering and verifying the low-confidence failure object, and re-determining the object category of the target object according to the verification result includes: Calculate the size ratio of the first position box and the second position box; In response to determining that the size ratio is within a preset ratio range, determine the low-confidence failure object as an object to be verified; Perform a relevance verification on the object to be verified, and determine the object category of the target object as the high-confidence valid object or the high-confidence failure object according to the verification result.

11. The method according to claim 9, further comprising: Determine a first position box of the low-confidence failure object in the first image; and The filtering and verifying the low-confidence failure object, and re-determining the object category of the target object according to the verification result includes: Calculate the occlusion area ratio according to the mapping relationship between the first position box and each segmentation region of the first image; In response to determining that the occlusion area ratio is within a preset ratio range, determine the low-confidence failure object as an object to be verified; Perform a relevance verification on the object to be verified, and determine the object category of the target object as the high-confidence valid object or the high-confidence failure object according to the verification result.

12. The method according to claim 10 or 11, wherein The performing a relevance verification on the object to be verified, and determining the object category of the target object as the high-confidence valid object or the high-confidence failure object according to the verification result includes: Calculate the similarity between the object to be verified in the first image and the same object in the database, and the similarity between the object to be verified in the second image and the same object in the database respectively, to obtain a third similarity and a fourth similarity; In response to determining that both the third similarity and the fourth similarity meet the preset similarity requirement, determine that the object category of the target object is the high-confidence valid object.

13. The method according to claim 12, further comprising: In response to determining that the third similarity and / or the fourth similarity do not meet the preset similarity requirement, determine that the object category of the target object is the high-confidence invalid object.

14. The method according to claim 13, wherein, The determining the status information of the target object according to the statistical result includes: In response to determining, according to the statistical result, that the number of times the object category of the target object is determined to be the high-confidence valid object is greater than a preset valid times threshold, determine that the status information of the target object is the valid status; In response to determining, according to the statistical result, that the number of times the object category of the target object is determined to be the high-confidence invalid object is greater than a preset invalid times threshold, determine that the status information of the target object is the invalid status; In response to determining, according to the statistical result, that the number of times the object category of the target object is determined to be the low-confidence invalid object is less than the invalid times threshold, determine that the status information of the target object is the unclear status.

15. The method according to claim 1, further comprising: In response to determining that the status information of the target object is the valid status, update the information of the point of interest corresponding to the target object in the map database; In response to determining that the status information of the target object is the invalid status, delete the information of the point of interest corresponding to the target object in the map database; In response to determining that the status information of the target object is the unclear status, generate a secondary acquisition task to re-determine the status information of the target object.

16. An apparatus for determining information validity, comprising: An image pair determination module, configured to pair the images in the first image set and the second image set according to the similarity between image features, to obtain at least one image pair, wherein the acquisition time of the second image set is earlier than the acquisition time of the first image set; A target object determination module, configured to determine the target object in the image pair according to the matching result of the image pixel points in the image pair; An object category determination module, configured to determine the object category of the target object in the image pair according to the feature similarity between the target object in the first image and the target object in the second image in the image pair; A status information determination module, configured to count the object categories of the target object in the at least one image pair, and determine the status information of the target object according to the statistical result, wherein the status information includes: valid status, invalid status and unclear status.

17. An electronic device, comprising: At least one processor; And A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method according to any one of claims 1-15.

18. A non-transitory computer-readable storage medium storing computer instructions for causing a computer to execute the method according to any one of claims 1-15.

19. A computer program product comprising a computer program which, when executed by a processor, implements the method according to any one of claims 1-15.

Images