Element recognition point data processing method and device, electronic equipment and storage medium

Abstract

The application provides an element recognition point data processing method and device, electronic equipment and a storage medium, relates to the technical field of electronic maps, can be applied to various scenes such as cloud technology, artificial intelligence, intelligent transportation and auxiliary driving, and the method comprises the following steps: receiving recognition point data of a road image reported by a terminal; performing initial aggregation processing on the recognition point data based on a time sequence to obtain a plurality of recognition point sequences; extracting a recognition point subsequence corresponding to an element label of at least one element type from each recognition point sequence of the plurality of recognition point sequences; determining a recognition target difference and an image time difference between two adjacent recognition point data in the recognition point subsequence; and performing same element aggregation on the recognition point data in the recognition point subsequence according to the recognition target difference, the image time difference and an element similarity condition to obtain a target sequence segment corresponding to the recognition point subsequence. The application can ensure the coverage and integrity of road elements and reduce the cost of the terminal.

Description

Technical Field

[0001] This application relates to the field of electronic map technology, and in particular to a method, apparatus, electronic device and storage medium for processing feature identification point data. Background Technology

[0002] Road features are an important component of map data. Typically, feature detection is performed on road images collected by the terminal to identify identical road features in the road images, thereby enabling map data production or updates. Due to the large number of road images collected by the terminal, full sampling recognition would consume extremely high resources for data transmission and recognition.

[0003] To reduce resource consumption, existing solutions employ terminal-reported trajectories, with the server segmenting the trajectories into fixed-length segments and using these segments as grouping criteria for identical road elements. Alternatively, high-precision cameras and highly intelligent detection models are used at the terminal to directly group identical road elements. However, the former approach involves cutting continuous trajectories of identical elements into different groups, resulting in insufficient completeness of the required data collection and failure to cover optimal identification points. Furthermore, grouping by trajectory points may result in the absence of valid identification points for road elements within each group, leading to data redundancy. The latter approach, on the other hand, has high terminal costs, stringent installation and usage requirements, and high computational resource demands. Summary of the Invention

[0004] This application provides a method, apparatus, device, and storage medium for processing feature identification point data.

[0005] On the one hand, this application provides a method for processing feature identification point data, applied to a server, the method comprising:

[0006] The receiving terminal reports road image recognition point data, which includes image time, trajectory point data associated with the road image, recognition targets of road elements in the road image, and element labels corresponding to the recognition targets. The element labels represent the element type of the road elements.

[0007] The identification point data is initially aggregated based on time sequence to obtain multiple identification point sequences. In two time-adjacent identification point sequences, the difference between the starting identification point data of the earlier identification point sequence and the ending identification point data of the later identification point sequence meets a preset difference condition.

[0008] Extract at least one subsequence of identification points corresponding to feature labels of at least one feature type from each of the plurality of identification point sequences;

[0009] The differences in the target recognition, image time, and trajectory point data between two adjacent recognition points in the recognition point subsequence are obtained.

[0010] Based on the differences in the identification targets, the differences in image time, and the similarity of elements, the identification point data in the identification point subsequence are aggregated for the same elements to obtain the target sequence segment corresponding to the identification point sequence. The identification point data in the target sequence segment belong to the same road element group.

[0011] On the other hand, a feature identification point data processing device is provided, applied to a server, the device comprising:

[0012] Identification point data receiving module: used to receive identification point data of road images reported by the terminal. The identification point data includes image time, trajectory point data associated with the road image, identification targets of road elements in the road image and element labels corresponding to the identification targets. The element labels represent the element type of the road elements.

[0013] First aggregation module: used to perform initial aggregation processing on the identification point data based on time sequence to obtain multiple identification point sequences. In two time-adjacent identification point sequences, the difference between the starting identification point data of the earlier identification point sequence and the ending identification point data of the later identification point sequence meets a preset difference condition.

[0014] Identification point sequence extraction module: used to extract identification point sequences corresponding to feature labels of at least one feature type from each of the plurality of identification point sequences;

[0015] Difference determination module: used to determine the difference in recognition target and the difference in image time between two adjacent recognition point data in the recognition point sub-sequence;

[0016] The second aggregation module is used to aggregate the identification point data in the identification point sub-sequence according to the identification target difference, image time difference and feature similarity conditions, to obtain the target sequence segment corresponding to the identification point sequence, wherein the identification point data in the target sequence segment belongs to the same road feature group.

[0017] On the other hand, a computer device is provided, the device including a processor and a memory, the memory storing at least one instruction or at least one program, the at least one instruction or the at least one program being loaded and executed by the processor to implement the feature identification point data processing method as described above.

[0018] On the other hand, a computer-readable storage medium is provided, wherein at least one instruction or at least one program is stored therein, the at least one instruction or the at least one program being loaded and executed by a processor to implement the feature identification point data processing method as described above.

[0019] On the other hand, a server is provided, the server including a processor and a memory, the memory storing at least one instruction or at least one program, the at least one instruction or the at least one program being loaded and executed by the processor to implement the feature identification point data processing method as described above.

[0020] On the other hand, a terminal is provided, which includes a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or the at least one program is loaded and executed by the processor to implement the feature identification point data processing method as described above.

[0021] On the other hand, a computer program product or computer program is provided, which includes computer instructions that, when executed by a processor, implement the feature identification point data processing method as described above.

[0022] The feature identification point data processing method, apparatus, equipment, storage medium, server, terminal, computer program, and computer program product provided in this application have the following technical effects:

[0023] The technical solution of this application receives road image recognition point data reported by the terminal on the server side. The recognition point data includes image time, trajectory point data associated with the road image, and recognition targets and corresponding element labels of road elements in the road image. The element labels represent the element type of the road element. The recognition point data is initially aggregated based on time sequence to obtain multiple recognition point sequences. In two temporally adjacent recognition point sequences, the difference between the starting recognition point data of the earlier recognition point sequence and the ending recognition point data of the later recognition point sequence meets a preset difference condition. Then, at least one subsequence of recognition points corresponding to element labels of at least one element type is extracted from each of the multiple recognition point sequences. The recognition target difference and image time difference between two adjacent recognition point data in the recognition point subsequence are determined. Then, the recognition point data in the recognition point subsequence are aggregated for the same element according to the recognition target difference, image time difference, and element similarity condition to obtain the target sequence segment corresponding to the recognition point sequence. The recognition point data in the target sequence segment belong to the same road element group. In this way, the server receives the identification point data reported by the terminal without needing to receive the overall image information. This reduces traffic and resource consumption while ensuring the coverage and integrity of road elements in the road image, avoiding missing the best identification point. Furthermore, it performs initial grouping by element labels that represent element types, and then further refines the grouping based on differences in identification targets and image time. This enables precise differentiation of data groups of the same road element. The terminal only needs to identify the identification target and element type, without needing to group the same road elements, thus reducing the terminal's data collection, identification, and equipment costs. Attached Figure Description

[0024] To more clearly illustrate the technical solutions and advantages in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 This is a schematic diagram of an application environment provided in an embodiment of this application;

[0026] Figure 2 This is a flowchart illustrating a method for processing feature identification point data provided in an embodiment of this application;

[0027] Figure 3 This is a flowchart illustrating another feature identification point data processing method provided in an embodiment of this application;

[0028] Figure 4 This is a flowchart illustrating another feature identification point data processing method provided in an embodiment of this application;

[0029] Figure 5 This is a schematic diagram illustrating the principle of generating an initial identification point sequence provided in an embodiment of this application;

[0030] Figure 6 These are four road images carrying feature recognition results corresponding to a recognition sub-sequence provided in this application embodiment;

[0031] Figure 7 These are four road images carrying feature recognition results corresponding to another identification sub-sequence provided in this application embodiment;

[0032] Figure 8 This is a schematic diagram of a set of target trajectory sequences, target sequence segments, and fused trajectory sequences provided in an embodiment of this application;

[0033] Figure 9 This is a schematic flowchart of a feature identification point data processing method provided in an embodiment of this application;

[0034] Figure 10 This is a schematic diagram of the flowchart of another feature identification point data processing method provided in the embodiments of this application;

[0035] Figure 11 This is a schematic diagram of the framework of a feature identification point data processing device provided in an embodiment of this application;

[0036] Figure 12 This is a hardware structure block diagram of an electronic device for a feature identification point data processing method provided in an embodiment of this application. Detailed Implementation

[0037] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0038] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or server that comprises a series of steps or sub-modules is not necessarily limited to those steps or sub-modules explicitly listed, but may include other steps or sub-modules not explicitly listed or inherent to such processes, methods, products, or devices.

[0039] Before providing a further detailed description of the embodiments of this application, the nouns and terms involved in the embodiments of this application will be explained, and the nouns and terms involved in the embodiments of this application shall be interpreted as follows.

[0040] Road elements: Common road markings, such as directional signs, speed limit signs, point speed limit signs, speed cameras, traffic lights, ground vehicle signals, hazard signs, road name signs, and other key information that map data collection and production focuses on.

[0041] Road data collection equipment refers to hardware devices installed in vehicles that can capture and record road information, including smart rearview mirrors, dashcams, in-vehicle cameras, or other hardware devices that can capture and record road information.

[0042] End-to-end acquisition: A new acquisition mode that uses real-time video stream detection algorithms deployed on road acquisition equipment to detect and acquire road elements.

[0043] Artificial intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess the functions of perception, reasoning, and decision-making.

[0044] Artificial intelligence (AI) is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies primarily include computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0045] Computer vision (CV) is the science that studies how to enable machines to "see." More specifically, it refers to machine vision, which uses cameras and computers to replace human eyes in recognizing and measuring targets, and then performs image processing to create images more suitable for human observation or transmission to instruments. As a scientific discipline, computer vision studies related theories and technologies, attempting to build artificial intelligence systems capable of extracting information from images or multidimensional data. Computer vision technologies typically include image processing, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content / behavior recognition, 3D object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping (SLAM), and common biometric recognition technologies such as facial recognition and fingerprint recognition.

[0046] In recent years, with the research and progress of artificial intelligence technology, artificial intelligence technology has been widely used in many fields. The solutions provided in the embodiments of this application involve artificial intelligence technologies such as machine learning / deep learning and natural language processing, which are specifically illustrated through the following embodiments.

[0047] Please see Figure 1 , Figure 1 This is a schematic diagram of an application environment provided in an embodiment of this application, such as... Figure 1 As shown, the application environment may include at least terminal 01 and server 02. In practical applications, terminal 01 and server 02 can be directly or indirectly connected via wired or wireless communication, and this application does not impose any restrictions on this.

[0048] In this application embodiment, server 02 can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms.

[0049] Specifically, cloud technology refers to a hosting technology that unifies hardware, software, and network resources within a wide area network (WAN) or local area network (LAN) to achieve data computation, storage, processing, and sharing. Cloud technology can be applied to various fields, such as medical cloud, cloud IoT, cloud security, cloud education, cloud conferencing, AI cloud services, cloud applications, cloud calling, and cloud social networking. Based on the cloud computing business model, cloud technology distributes computing tasks across a resource pool composed of numerous computers, enabling various application systems to obtain computing power, storage space, and information services as needed. The network providing these resources is called the "cloud." From the user's perspective, the resources in the "cloud" are infinitely scalable, readily available, on-demand, expandable, and pay-as-you-go. As a provider of basic cloud computing capabilities, a cloud resource pool (referred to as a cloud platform, generally called IaaS (Infrastructure as a Service)) platform is established, deploying various types of virtual resources within the pool for external customers to choose from. The cloud resource pool mainly includes: computing devices (virtualized machines containing operating systems), storage devices, and network devices.

[0050] Specifically, the server 02 mentioned above may include physical devices, such as network communication submodules, processors, and memory, and may also include software running on the physical devices, such as applications.

[0051] Specifically, terminal 01 may include physical devices such as smartphones, desktop computers, tablets, laptops, digital assistants, augmented reality (AR) / virtual reality (VR) devices, smart voice interaction devices, smart home appliances, smart wearable devices, and in-vehicle terminal devices, and may also include software running on the physical devices, such as applications.

[0052] In this embodiment, terminal 01 can be used to send identification point data of feature identification points to server 02, and can also be used to send sampling trajectory point data of interval sampling to server 02. Server 02 can be used to provide initial aggregation processing of identification point data, feature extraction, data difference determination, and aggregation of the same features to obtain target sequence segments, where the identification point data in the target sequence segments belong to the same road feature group; and can also be used to splice the sampling trajectory point data and determine the sampling trajectory sequence that matches the target sequence segment, thereby generating a fused trajectory sequence; in addition, it can also determine the target identification point data in each target sequence segment, and determine the frame-cutting timestamp in combination with the fused trajectory sequence, whereby the frame-cutting timestamp is used to indicate the time point at which the terminal device performs frame-cutting operation on the video stream of the road image.

[0053] Furthermore, it is understandable that Figure 1 The example shown is merely an application environment for a feature identification point data processing method. This application environment may include more or fewer nodes, and this application does not impose any limitations on it.

[0054] The following describes a feature identification point data processing method based on the aforementioned application environment. This method is applied to a server, and its embodiments can be applied to various scenarios, including but not limited to cloud technology, artificial intelligence, smart transportation, and assisted driving. Please refer to... Figure 2 , Figure 2 This is a flowchart illustrating a method for processing feature identification point data according to an embodiment of this application. This specification provides method operation steps as shown in the embodiments or flowcharts, but based on conventional or non-inventive labor, more or fewer operation steps may be included. The order of steps listed in the embodiments is merely one possible execution order among many and does not represent the only execution order. In actual system or server product execution, the method can be executed sequentially according to the embodiments or drawings, or in parallel (e.g., in a parallel processor or multi-threaded processing environment). Specifically, as... Figure 2 As shown, the method may include the following steps S201-S209.

[0055] S201: Receive the identification point data of the road image reported by the terminal.

[0056] In this embodiment, the identification point data is obtained by the terminal performing element recognition on a video stream containing continuous road images, and then combining the obtained element recognition results with trajectory point data associated with the road images. Obtaining identification point data instead of acquiring uploaded road images from the terminal reduces data acquisition traffic, lowers bandwidth usage, and reduces data processing resource consumption.

[0057] Specifically, the identification point data includes image time, trajectory point data associated with the road image, and identification targets and corresponding element labels for road elements in the road image. Specifically, image time represents the image acquisition time or image reporting time of the road image, such as the acquisition timestamp of the road image. Specifically, trajectory point data includes, but is not limited to, trajectory point coordinates, and may also include, but is not limited to, trajectory point timestamps or signal network types. The trajectory point timestamp can be the acquisition timestamp or reporting timestamp of the trajectory point data, and the trajectory point coordinates can be the latitude and longitude of the trajectory point. The trajectory point data associated with the road image can be trajectory point data whose trajectory point timestamps match the image time. Matching the trajectory point timestamps and image time can mean that the times are consistent or the time difference is within a certain range.

[0058] Specifically, road elements are common road marking information, such as, but not limited to, driving direction markings, traffic speed limit signs, point speed limit markings, speed cameras, traffic lights, ground vehicle signals, hazard signs, road name signs, and other key information that map data collection and production focuses on; identification targets include the bounding box coordinates of the road element's identification box in the road image. The identification box is used to define and mark the pixel range of the road element in the road image, and can be, for example, a bounding box (bbox); element labels represent the element type of the road element. Each identification target corresponds to an element label, such as a traffic light representing a road element identified in the road image and marked by an identification box.

[0059] In this embodiment, multiple road acquisition devices installed on a vehicle can acquire video streams including road images during vehicle operation. A local real-time video stream detection algorithm is then used to detect and acquire these video streams, obtaining image time and feature recognition results. The feature recognition results include the identified target and identification labels, and may also include confidence scores. The real-time video stream detection algorithm can be implemented based on a feature recognition model. The terminal uses this model to identify road features in the video stream, obtaining feature recognition results. The feature recognition model can be built based on algorithms such as Fully Convolutional One-Stage Object Detection (FCOS). It should be noted that the identified target does not need to include the specific character content of the road features within the recognition box, nor does it need to identify the same road features in different road images. For example, it does not need to identify the specific road in the road sign, nor does it need to uniformly label the same road signs in different images. This reduces the computing power and resource requirements of various types of acquisition terminals, lowering the cost of terminal equipment.

[0060] In one embodiment, the feature recognition result includes the bounding box (bbox) of the road feature, the feature label, and the confidence score, with the data format as follows:

[0061] "x1":10, #coordinates of the top left corner of the bbox

[0062] "y1":10, #bbox coordinates of the top right and top left corners

[0063] "x2":30, #bbox bottom right corner coordinates

[0064] "y2":30, #bbox bottom right corner coordinates

[0065] "label":1,#type

[0066] "score": 90.0, #confidence level

[0067] In this embodiment, the terminal can also collect driving trajectories through a locally installed road trajectory acquisition device. The road trajectory acquisition device may include onboard hardware and software, or only software, such as a client communicating with the trajectory acquisition server. The driving trajectory consists of trajectory points and can be obtained using a positioning system, such as GPS. After recognizing the feature recognition result of a road image, the terminal can determine the associated trajectory points from the driving trajectory based on the image time (e.g., image timestamp) of the road image, thereby obtaining associated trajectory point data. The trajectory point timestamps of the associated trajectory points match the image timestamps. Further, the terminal combines the image time, associated trajectory point data, and feature recognition results to obtain recognition point data, and sends the recognition point data to the server.

[0068] In one embodiment, the data format of the identification point data is as follows:

[0069] "x1":10, #coordinates of the top left corner of the bbox

[0070] "y1":10, #bbox coordinates of the top right and top left corners

[0071] "x2":30, #bbox bottom right corner coordinates

[0072] "y2":30, #bbox bottom right corner coordinates

[0073] "label":1,#type

[0074] "score": 90.0, #confidence level

[0075] "P time stamp":XXXX,#image timestamp

[0076] "TrackPoint":(x i ,y i ), # Coordinates of trajectory points

[0077] "T time stamp":XXXX,#Trajectory point timestamp

[0078] Understandably, a single road image may include multiple road elements, and correspondingly, the identification point data includes the identification targets and element labels for each of the multiple road elements.

[0079] Understandably, the identification point data sent by the terminal also includes the unique device identifier of the terminal device to facilitate data differentiation.

[0080] S203: Perform initial aggregation processing on the identification point data based on time sequence to obtain multiple identification point sequences.

[0081] In this embodiment of the application, in two temporally adjacent identification point sequences, the difference between the starting identification point data of the earlier identification point sequence and the ending identification point data of the later identification point sequence satisfies a preset difference condition.

[0082] In practical applications, S203 may include the following S2031-S2032.

[0083] S2031: Aggregate the identification point data based on time sequence and preset difference conditions to obtain multiple initial identification point sequences.

[0084] Specifically, the temporal sequence here can refer to the sorting of image timestamps. Based on the image timestamps in each recognition point data point, the data points are sorted temporally, preferably from earliest to latest. Then, aggregation is performed based on preset difference conditions. The resulting initial recognition point sequence contains more than or equal to a preset number of recognition points, such as 300. Please refer to [reference needed]. Figure 5 Sorting images based on time can prevent the loss of identification point data that is delayed in reporting within a certain period of time, thus ensuring data integrity.

[0085] Specifically, the preset difference condition may include: the image time difference between the starting identification point data of an earlier identification point sequence and the ending identification point data of a later identification point sequence is greater than or equal to a first preset value, which may be, for example, 10 seconds. In this way, the large amount of identification point data reported by the terminal is aggregated to achieve initial grouping of identification points, thereby appropriately reducing the amount of data in a single grouping of the same elements.

[0086] In some cases, the process of sequentially sorting the data at each identification point may also include a step of filtering out identification point data that has timed out. Specifically, a timeout occurs when the image time difference between one or more identification point data and subsequent identification point data exceeds a second preset value, which may be, for example, 1 hour. Please refer to [reference needed]. Figure 5In the figure, the image time of the overdue identification point data 8 is 8:00 am on the same day, while the image time of the adjacent identification point data 20 is 9:30 am on the same day. Therefore, identification point data 8 is determined to be overdue and is discarded.

[0087] S2032: Filter out abnormal identification points from multiple initial identification point sequences to obtain an identification point sequence.

[0088] In specific embodiments, abnormal identification point detection is performed on each of the multiple initial identification point sequences to filter out abnormal identification points. In some embodiments, if the distance between the trajectory points of two consecutive identification point data with the same feature label in the initial identification point sequence is within a preset distance range, and the rate of change of the identification target is within a preset rate of change range, the later identification point data in the two identification point data is determined as an abnormal identification point; and then the abnormal identification point is removed from the initial identification point sequence. For example, in the initial identification point sequence [a / b / c / d / e / f…], if the feature label of identification point data b is different from that of identification point data c, but the feature label is the same as that of identification point data e, then b and e are determined to be two consecutive identification point data with the same feature label.

[0089] Specifically, identical element labels mean that the identification point data of the two road images include the same number of identification targets and the element labels of each identification target are the same. For example, both identification point data include 3 identification targets and the element labels of the three identification targets are road name sign, road name sign and traffic light. The trajectory point distance can be calculated based on the trajectory point coordinates in the two identification point data, and the identification target change rate can be the identification box area change rate calculated based on the box coordinates of the two identification point data.

[0090] Specifically, the distance between trajectory points of two consecutive identification points with the same feature label in the initial identification point sequence can be determined first. If the distance between trajectory points is within a preset distance range, the rate of change of the identification target between the two identification point data can be calculated. If the rate of change of the identification target is within a preset rate of change range, an abnormal identification point is identified. Otherwise, it is determined that neither of the two identification point data is an abnormal identification point.

[0091] Specifically, multiple preset distance ranges and multiple preset change rate ranges can be set, with each preset distance range corresponding to a preset change rate range. For example, a preset distance range may be greater than or equal to 200 meters, with a preset change rate less than or equal to 5%, or a preset distance range may be less than or equal to 50 meters, with a preset change rate greater than or equal to 50%. In one embodiment, if the distance between trajectory points exceeds 200 meters and the change rate of the identified target is within 5%, then the later identified point data is determined to be an abnormal identified point; or, if the distance between trajectory points is less than 50 meters and the change rate of the identified target exceeds 50%, then the later identified point data is determined to be an abnormal identified point.

[0092] S205: Extract at least one subsequence of identification points corresponding to feature labels of at least one feature type from each of the multiple identification point sequences.

[0093] In this embodiment, the element labels for all element types corresponding to each identification point data in the identification point sequence can be statistically determined. For example, if a certain identification point sequence corresponds to three element types of element labels, namely road name signs, traffic lights, and lane lines, then for at least one element type, identification point data with at least one element label is extracted from the identification point sequence. For example, identification point data with the element label of traffic lights is extracted to obtain identification point subsequence 1; identification point data with the element label of lane lines is extracted to obtain identification point sequence 2; and identification point data with the element label of road name signs is extracted to obtain identification point sequence 3. It is understood that the same identification point data may include two or more element labels. For example, identification point data a includes both road name signs and lane lines as element labels, then identification point data a is included in both identification point subsequences 2 and 3. In some embodiments, identification point subsequences corresponding to the element labels of each element type are extracted from each identification point sequence of multiple identification point sequences, that is, identification point subsequences are extracted separately for all element types to improve the utilization rate of identification point data.

[0094] S207: Determine the differences in the target recognition and the differences in image time between two adjacent recognition points in the recognition point subsequence.

[0095] In this embodiment, the difference in identification targets refers to the difference between the identification targets corresponding to the element labels extracted from the identification point subsequences to which two identification point data belong. For example, the element label extracted from identification point subsequence 3 is a road sign, and the difference in identification targets refers to the difference between the identification targets corresponding to the road signs in the two identification point data in identification point subsequence 3.

[0096] S209: Based on the differences in the identification targets, the differences in image time, and the similarity of elements, the identification point data in the identification point subsequence are aggregated to obtain the target sequence segment corresponding to the identification point sequence.

[0097] In this embodiment, the identification point data in the target sequence segment belong to the same road element group. Thus, by leveraging the acquisition capabilities of low-end terminal devices and the backhaul of vector data from basic road element perception detection and classification, high-quality element grouping can be performed in the cloud to achieve high ROI image data recovery.

[0098] In practical applications, S209 may include S2091-S2092.

[0099] S2091: If either the target difference or the image time difference does not meet the element similarity condition, determine that the data of two adjacent identification points belong to different road elements.

[0100] S2092: Group and segment adjacent identification point data belonging to different road elements in the identification point subsequence to obtain the target sequence segment corresponding to the identification point sequence.

[0101] In a specific embodiment, the difference in the identified targets includes the reduction in the number of identified targets and the reduction rate of the identified target area. The reduction rate of the identified target area represents the reduction ratio of the identified targets in the later identified point data compared to the identified targets in the earlier identified point data among two adjacent identified point data. Correspondingly, the element similarity condition includes at least one of the following sub-conditions: the reduction rate of the identified target area is less than or equal to a ratio threshold; the image time difference is less than or equal to a time difference threshold; and the reduction in the number of identified targets is less than a quantity threshold. Specifically, if at least one of the following conditions is met—that the target area reduction rate is greater than a ratio threshold, the image time difference is greater than a time difference threshold, and the reduction in the number of identified targets is greater than or equal to a quantity threshold—then it is determined that the two identified point data belong to different road elements.

[0102] Specifically, the target area reduction rate here refers to the area reduction rate between two identification point data sets corresponding to the feature labels of their respective identification point subsequences, and the reduction in the number of identification targets refers to the reduction in the number of identification targets between two identification point data sets corresponding to the feature labels of their respective identification point subsequences. In some embodiments, the identification target includes the bounding box coordinates of the identification box, then the above-mentioned target area reduction rate is the identification box area reduction rate, and the reduction in the number of identification targets is the identification box reduction rate. In one embodiment, the feature label of the extracted identification subsequence is feature label K, and the identification box area reduction rate P can be calculated using the following formula, where i represents the identifier of the identification box in the earlier identification point data of the two identification point data sets, j represents the identifier of the identification box in the later identification point data of the two identification point data sets, and N... m N represents the number of bounding boxes for feature label K in the earlier identification point data. m+1 X represents the number of bounding boxes for feature label K in later identification point data. i1 X is the x-coordinate of the top-left corner of the bounding box i. i2 It is the x-coordinate of the bottom right corner of the recognition box, Y. i1 Y is the ordinate of the top-left corner of the bounding box i. i2 It is the y-coordinate of the bottom right corner of the bounding box i, and similarly, X j1 X j2 Y j1 Y j2 These are the x-coordinates of the top left corner, the x-coordinate of the bottom right corner, the y-coordinate of the top left corner, and the y-coordinate of the bottom right corner of the recognition box j, respectively.

[0103]

[0104] For example, the feature label corresponding to the identification point sub-sequence is a road sign, the ratio threshold can be, for example, 30%, the time difference threshold can be, for example, 10s, and the quantity threshold can be, for example, 1. Based on the time sequence from early to late, if the reduction rate of the total recognition area of ​​the road sign recognition boxes in the subsequent identification point data is greater than 30% compared to the total recognition box area of ​​the road sign recognition boxes in the previous identification point data, or if the reduction in the number of recognition boxes of road sign recognition boxes in the subsequent identification point data is greater than or equal to 1 compared to the number of recognition boxes of road sign recognition boxes in the previous identification point data (i.e., more than one road sign recognition box has disappeared), or if the image time difference between the preceding and following identification point data exceeds 10s, it is determined that the two identification point data do not belong to the same road feature; then, the two identification point data are segmented to allocate them to different target sequence segments. It can be understood that each identification sub-sequence can obtain at least one target sequence segment. If r pairs of identification data points belonging to different road features are determined, then the identification sub-sequence corresponds to r+1 target sequence segments.

[0105] Please refer to Figure 6 The four road images from top to bottom in the figure include road signs and lane lines, with corresponding element labels of "road sign" and "lane line". For road signs, the total recognition box area of ​​the road signs in the first three images gradually increases in size from far to near. Although the fourth image also contains road signs, the reduction rate of the total recognition box area far exceeds 30%. Therefore, the first three images belong to the same road element interval, i.e., the same target sequence segment, while the fourth belongs to the next target sequence segment. The same grouping result can be obtained based on the lane lines. Please refer to [the image / reference]. Figure 7 The element label in the figure is road sign. The recognition boxes of the first three road signs are from small to large. The recognition box of the last sign is reduced by more than 30% compared to the recognition box of the third sign. Therefore, the first three signs belong to the same target sequence segment, and the last sign belongs to another target sequence segment.

[0106] Specifically, the server can send the sequence information of the target sequence segment to the terminal. The sequence information can include the image time of each recognition point data in the target sequence segment, so that the terminal can group the road images in the video stream based on the image time, and map the grouping information of the road elements to the road image grouping of the terminal to obtain multiple road image groups containing the same road element.

[0107] Understandably, the identification point data is ordered chronologically. During vehicle movement, the distance between the vehicle and the same road element increases from far to near, until the road element exceeds the field of view of the road acquisition equipment on the vehicle. Correspondingly, the imaging area of ​​the same road element in continuous road images decreases in size as it moves away and increases in size as it moves closer, until it disappears. Therefore, based on the reduction rate of the target area and the decrease in the number of identification points, it is possible to directly identify whether two identification point data belong to the same road element group under the current element label. Thus, by performing simple calculations based on the changing patterns of the image and recognition results during image capture, different road element groups can be determined, ensuring grouping accuracy while reducing resource consumption and improving grouping efficiency.

[0108] Specifically, the target sequence segment can be associated with the device's unique identifier and stored together. Additionally, the target sequence segment can be associated with the element label of its corresponding element type and stored together. For example, if the identification subsequence is extracted based on the element label of the road name sign, then the corresponding target sequence segment can be associated with the element label - road name sign and stored together.

[0109] In summary, the technical solution of this application receives the identification point data reported by the terminal on the server side without needing to receive the overall image information. This reduces traffic and resource consumption while ensuring the coverage and integrity of road elements in the road image, avoiding the omission of the best identification points. Furthermore, it uses element labels representing element types for initial grouping, and then further refines the grouping based on differences in the identification target and image time, accurately distinguishing data groups of the same road element. The terminal only needs to identify the identification target and element type, eliminating the need for grouping identical road elements, thus reducing terminal acquisition, identification, and equipment costs. Moreover, grouping the continuously reported identification point data from the terminal device into element groups for acquisition operations ensures the continuity of road operations; and grouping identification point data of the same road element into the same group saves acquisition costs, avoids multiple acquisitions of the same element, and facilitates road element management at the element label level.

[0110] Based on some or all of the above implementation methods, please refer to Figure 3 In this embodiment of the application, the method further includes the following steps S301-S307.

[0111] S301: Receive the sampling trajectory point data reported by the terminal during interval sampling.

[0112] As mentioned earlier, the terminal can collect driving trajectories through a locally installed road trajectory acquisition device. These trajectories reflect the real-time position changes of the vehicle during its journey and include a large number of trajectory points. The terminal can perform interval sampling of the trajectory points in the driving trajectory and then report the obtained sampled trajectory point data to the server. The sampling interval can be a time interval or a number of trajectory points interval, such as sampling and reporting one trajectory point data every 1 second, or sampling and reporting one trajectory point data every 10 trajectory points.

[0113] S303: Based on the time sequence, the sampled trajectory point data are spliced ​​together to obtain a sampled trajectory sequence set.

[0114] In practical applications, S303 can include S3031-S3032.

[0115] S3031: Based on the time sequence and preset segmentation interval, the sampled trajectory point data is segmented and spliced ​​to obtain an initial sampled trajectory sequence set.

[0116] S3032: Perform deduplication on each initial sampled trajectory sequence in the initial sampled trajectory sequence set to obtain a sampled trajectory sequence set.

[0117] Specifically, the preset segmentation interval can be a preset number of segmented trajectory points, a preset segmentation duration, or a preset segmentation distance, etc. If it is a preset number of segmented trajectory points, then based on the time sequence, the preset number of consecutive sampled trajectory point data are spliced ​​together, and each initial sampled trajectory sequence in the initial sampled trajectory sequence set includes the preset number of sampled trajectory point data. If it is a preset segmentation duration, then the time difference between the first and last sampled trajectory point data of each initial sampled trajectory sequence is less than or equal to a preset time. If it is a preset segmentation distance, then the trajectory point distance between the first and last sampled trajectory point data of each initial sampled trajectory sequence is less than or equal to a preset distance.

[0118] Specifically, deduplication refers to removing duplicate sampling trajectory points from the initial sampling trajectory sequence, i.e., sampling trajectory points with the same coordinates (such as latitude and longitude) or the same timestamp, to obtain the sampling trajectory sequence. The sampling trajectory sequence set is then associated and stored with the unique device identifier of the terminal device. Furthermore, each initial sampling trajectory sequence can be associated and stored with its corresponding trajectory start time.

[0119] S305: Determine the target trajectory sequence that matches the target sequence segment from the sampled trajectory sequences in the sampled trajectory sequence set.

[0120] In practical applications, a target trajectory sequence refers to the sampled trajectory sequence that matches the image times of the starting and ending recognition point data within the sampled trajectory sequence set. The time period corresponding to the target sequence segment is within the trajectory time period corresponding to the target trajectory sequence, which refers to the time period formed by the image times of its first and last recognition point data. It is understandable that the database stores sampled trajectory sequences uploaded by multiple terminal devices. Therefore, when determining the target trajectory sequence, it is necessary to first determine the sampled trajectory sequence set belonging to the same terminal device as the current target sequence segment based on the device's unique identifier, and then perform a match based on image time.

[0121] In some cases, the target trajectory sequence must satisfy a certain time difference or distance between its first trajectory point data and the trajectory point data of the starting identification point data in the target sequence segment, and between the trajectory point data of the last trajectory point data in the target sequence segment and the trajectory point data of the target trajectory sequence that must satisfy the last identification point data. For example, the first trajectory point data of the target trajectory sequence must be 3 seconds earlier than the trajectory point data of the starting identification point data in the target sequence segment, and the last trajectory point data in the target sequence segment must be 3 seconds earlier than the trajectory point data of the target trajectory sequence that must satisfy the last identification point data. The specific settings can be configured according to actual needs. Furthermore, during the hit process, temporally adjacent sampled trajectory sequences can be acquired, spliced, and truncated to obtain the target trajectory sequence.

[0122] S307: Based on the time series, the target sequence segment and the target trajectory sequence are fused to obtain the fused trajectory sequence.

[0123] In practical applications, the target sequence segment is merged with the target trajectory sequence to form a fused trajectory fragment. Specifically, the identification point data is interpolated into the target trajectory sequence based on temporal or positional order to obtain the fused trajectory sequence. Please refer to [reference needed]. Figure 8 The first row in the diagram represents the target trajectory sequence, the second row represents the target sequence segments, and the third row represents the fused trajectory sequence obtained through fusion processing. This fused trajectory sequence can then be stored in the master map database for subsequent map data production and updates. The fused trajectory sequence can be stored in association with unique device identifiers and feature labels.

[0124] Based on some or all of the above implementation methods, please refer to Figure 4 In this embodiment of the application, the method further includes the following steps S401-S405.

[0125] S401: Determine the target identification point data from the identification point data in the target sequence segment.

[0126] In practical applications, the identification target includes the bounding box coordinates of the road element in the road image; correspondingly, the target identification point data can be determined by: determining the area of ​​the bounding box corresponding to each identification point data in the target sequence segment based on the bounding box coordinates; and determining the identification point data with the largest bounding box area in the target sequence segment as the target identification point data.

[0127] Understandably, the recognition box area here refers to the total area of ​​the recognition boxes corresponding to the feature labels associated with the target sequence segment. For example, if the feature label corresponding to the target sequence segment is a road name sign, then the recognition box area is the total area of ​​the recognition boxes of each road name sign in the recognition point data. The recognition point data with the largest area is determined as the target recognition point data. The target recognition point data is the best recognition point for recognizing its corresponding road features. The road image of this target recognition point data is the best recognition image among all the road images corresponding to the target sequence segment for recognizing its road features (such as the aforementioned road name sign recognition).

[0128] S403: Determine the frame timestamp corresponding to the fusion trajectory sequence based on the image time and the frame point selection conditions corresponding to the feature labels of the target recognition point data.

[0129] S405: Send the frame capture timestamp to the terminal so that the terminal can filter out road images whose image time matches the frame capture timestamp.

[0130] In practical applications, frames can be cropped based on different feature types, that is, frames can be cropped for fused trajectory sequences of different feature types. Specifically, image return tasks can be received, including but not limited to the saved feature tags (i.e., feature types) and device unique identifiers that need to be cropped. Based on the feature tags and device unique identifiers, multiple matching fused trajectory sequences can be read from storage. Then, based on the image time of the target identification point data and the frame cropping point filtering conditions, the time point for cropping in each fused trajectory sequence is determined, i.e., the frame cropping timestamp is obtained. This timestamp is sent to the terminal corresponding to the device unique identifier, so that the terminal can crop and save the video stream and the road image corresponding to the frame cropping timestamp, and then send it back to the server for storage for subsequent map data production.

[0131] In some embodiments, the server can generate a frame capture order, which includes a frame capture timestamp and a unique device identifier, as well as an image return address. After receiving the frame capture order, the terminal will find the specific location in the video file based on the specific timestamp, capture the video frame, and return it.

[0132] Specifically, each feature type's feature label can correspond to a frame point filtering condition. For example, the frame point filtering condition could be to use the timestamp corresponding to the image time of the target identification point data as a frame timestamp; or it could be to use the timestamp corresponding to the image time of the target identification point data as the central frame timestamp, and then sample a certain number of timestamps at equal intervals or at equal times before and after this central frame timestamp as auxiliary frame timestamps. For instance, to capture frames of traffic light features, in addition to the target identification point data, the frame points would need to capture 5 road images each at equal intervals of 6 meters forward and backward, resulting in 11 images (5+1+5).

[0133] Therefore, the map feature data acquisition process needs to be refined for feature acquisition. The most critical part of the acquisition process is to obtain the best identification point and the continuous positional changes before and after the best identification point. The above solution is to group the feature identification point data reported by a single device during its operation, group feature identification points containing the same road features into the same group, extract the best identification point, and extract the timestamps before and after the best identification point to generate an order for issuance, so as to facilitate the acquisition and retrieval of road images.

[0134] The following combination Figure 9 and Figure 10 The process framework described in this application introduces the feature identification point data processing method, such as... Figure 9 As shown, the process framework is as follows: 1. Receive identification point data; 2. Group elements to obtain target sequence segments; 3. Perform optimal point identification on the grouped elements; 4. Merge trajectory elements to obtain fused trajectory sequences; 5. Query / lock tasks to query image feedback tasks and lock the tasks for execution; 6. Perform strategy frame capture to determine the frame capture timestamp; 7. Generate and issue element frame capture orders; 8. Receive the returned road images and store them in the acquisition library.

[0135] Further, please refer to Figure 10 The specific processing steps include:

[0136] S1. Receive trajectory reports to obtain sampled trajectory point data;

[0137] S2. Trajectory splicing to obtain an initial set of sampled trajectory sequences;

[0138] S3. Trajectory preprocessing to obtain a set of sampled trajectory sequences;

[0139] S4. Store the sampled trajectory sequence set into the trajectory pool;

[0140] S5. Receive identification point data;

[0141] S6. Feature aggregation to obtain an initial sequence of identification points;

[0142] S7. Element delay judgment to remove identification point data that has been reported out of time;

[0143] S8. Anomaly detection to filter out anomaly detection points and obtain a sequence of detection points;

[0144] S9. Calculate the differences in recognition point data, including differences in recognition targets, differences in image time, and differences in distance between trajectory points, etc.

[0145] S10. Extract labels of elements of the same category to obtain a sequence of identification points;

[0146] S11. Group the elements to obtain the target sequence segment;

[0147] S12. Element + Trajectory Fusion: This involves fusing the target sequence segment with the target sampled sequence in the sampled trajectory sequence set of S4 to obtain a fused trajectory sequence.

[0148] S13. Element order generation, element order frame timestamp.

[0149] In summary, the technical solution of this application can be applied to the process of terminal devices reporting identification point data to the server. It performs feature detection on the reported vector data, initial aggregation processing, sequence extraction for each type of element label, and calculation of differences between consecutive element identification point data. This allows identification point data for the same road element to be grouped into the same group, achieving an accuracy rate of over 90%, avoiding chaotic collection of element information, and enabling the selection of optimal identification points. Furthermore, by fusing the grouped identification point data with the trajectory point data reported by the terminal, a strategy-based frame capture is performed. The device then captures specific timestamps and sends them to the terminal to obtain corresponding returned images for subsequent production and updating of specific element information. This enables centralized processing of element identification point data, improves the completeness of road element coverage in the returned images, achieves initial image classification, and improves the quality and efficiency of map data production. Moreover, this solution can be applied to terminals with relatively low device performance and computing power, reducing terminal cost requirements and ensuring the large-scale application of crowdsourced road element-level data collection.

[0150] This application embodiment also provides a feature identification point data processing device 500, such as... Figure 11 As shown, Figure 11 The diagram shows a structural schematic of a feature identification point data processing device provided in an embodiment of this application. The device may include the following modules.

[0151] Identification point data receiving module 10: used to receive identification point data of road images reported by the terminal. The identification point data includes image time, trajectory point data associated with the road image, and identification targets of road elements in the road image and the element labels corresponding to the identification targets. The element labels represent the element type of the road elements.

[0152] First aggregation module 20: used to perform initial aggregation processing on the identification point data based on time sequence to obtain multiple identification point sequences. In two time-adjacent identification point sequences, the difference between the starting identification point data of the earlier identification point sequence and the ending identification point data of the later identification point sequence meets the preset difference condition.

[0153] Identification point sequence extraction module 30: used to extract identification point sequences corresponding to feature labels of at least one feature type from each identification point sequence of multiple identification point sequences;

[0154] Difference determination module 40: used to determine the difference in recognition target and image time between two adjacent recognition point data in the recognition point subsequence;

[0155] The second aggregation module 50 is used to aggregate the identification point data in the identification point subsequence according to the identification target difference, image time difference and feature similarity conditions, to obtain the target sequence segment corresponding to the identification point sequence, and the identification point data in the target sequence segment belongs to the same road feature group.

[0156] In some embodiments, the second aggregation module 50 includes:

[0157] Conditional judgment unit: If either the target difference or the image time difference does not meet the element similarity condition, it determines that two adjacent identification point data belong to different road elements.

[0158] Segmentation unit: Used to group and segment adjacent identification point data belonging to different road elements in the identification point subsequence to obtain the target sequence segment corresponding to the identification point sequence.

[0159] In some embodiments, the difference in the number of identified targets includes the reduction in the number of identified targets and the reduction rate of the area of ​​identified targets. The reduction rate of the area of ​​identified targets represents the reduction ratio of the identified targets in the later identified point data compared to the identified targets in the earlier identified point data among two adjacent identified point data. The element similarity condition includes at least one of the following sub-conditions:

[0160] The target area reduction rate is less than or equal to the ratio threshold;

[0161] The image temporal difference is less than or equal to the temporal difference threshold;

[0162] The reduction in the number of identified targets is less than the quantity threshold.

[0163] In some embodiments, the first aggregation module 20 includes:

[0164] Data aggregation unit: used to aggregate identification point data based on time series and preset difference conditions to obtain multiple initial identification point sequences;

[0165] Identification point filtering unit: used to filter out abnormal identification points in multiple initial identification point sequences to obtain identification point sequences.

[0166] In some embodiments, the identification point filtering unit includes:

[0167] Anomaly point identification subunit: If, in the initial identification point sequence, the distance between the trajectory points of two consecutive identification point data with the same feature label is within a preset distance range, and the rate of change of the identified target is within a preset rate of change range, the later identification point data among the two identification point data is identified as an anomaly identification point.

[0168] Anomaly removal subunit: Used to remove anomaly identification points from the initial identification point sequence.

[0169] In some embodiments, the apparatus further includes:

[0170] Track point receiving module: used to receive the sampled track point data of the terminal reported by the interval sampling;

[0171] The trajectory stitching module is used to stitch together sampled trajectory point data based on time sequence to obtain a set of sampled trajectory sequences.

[0172] Target trajectory sequence determination module: used to determine the target trajectory sequence that matches the target sequence segment from the sampled trajectory sequences of the sampled trajectory sequence set;

[0173] Fusion processing module: used to fuse the target sequence segment and the target trajectory sequence based on time sequence to obtain the fused trajectory sequence.

[0174] In some embodiments, the apparatus further includes:

[0175] Target data determination module: used to determine target identification point data from the identification point data in the target sequence segment;

[0176] Frame capture time determination module: used to determine the frame capture timestamp corresponding to the fusion trajectory sequence based on the image time of the target recognition point data and the frame capture point filtering conditions corresponding to the feature labels;

[0177] Data sending module: Used to send the frame timestamp to the terminal so that the terminal can filter out road images whose image time matches the frame timestamp.

[0178] In some embodiments, the target identification includes the bounding box coordinates of road features in a road image; the target data determination module includes:

[0179] Frame area determination unit: used to determine the area of ​​the recognition frame corresponding to each recognition point data in the target sequence segment based on the frame coordinates;

[0180] Identification point data determination unit: used to determine the identification point data with the largest identification box area in the target sequence segment as the target identification point data.

[0181] In some embodiments, the trajectory stitching module includes:

[0182] Trajectory segmentation and splicing unit: used to segment and splice the sampled trajectory point data based on time sequence and preset segmentation interval to obtain an initial sampled trajectory sequence set;

[0183] Trajectory deduplication unit: Used to deduplicat each initial sampled trajectory sequence in the initial sampled trajectory sequence set to obtain a sampled trajectory sequence set.

[0184] It should be noted that the above-described device embodiments and method embodiments are based on the same implementation methods.

[0185] This application provides a feature identification point data processing method device. The scheduling device can be a terminal or a server, including a processor and a memory. The memory stores at least one instruction or at least one program. The at least one instruction or at least one program is loaded and executed by the processor to implement the feature identification point data processing method provided in the above method embodiments.

[0186] The memory can be used to store software programs and modules. The processor executes various functional applications and feature identification point data processing methods by running the software programs and modules stored in the memory. The memory can mainly include a program storage area and a data storage area. The program storage area can store the operating system, application programs required for the functions, etc.; the data storage area can store data created according to the use of the device, etc. In addition, the memory can include high-speed random access memory, and can also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory can also include a memory controller to provide the processor with access to the memory.

[0187] The methods and embodiments provided in this application can be executed in electronic devices such as mobile terminals, computer terminals, servers, or similar computing devices. Figure 12 This is a hardware structure block diagram of an electronic device for a feature identification point data processing method provided in an embodiment of this application. For example... Figure 12As shown, the electronic device 900 can vary significantly due to different configurations or performance. It may include one or more Central Processing Units (CPUs) 910 (CPUs 910 may include, but are not limited to, microprocessors such as MCUs or programmable logic devices such as FPGAs), a memory 930 for storing data, and one or more storage media 920 (e.g., one or more mass storage devices) for storing application programs 923 or data 922. The memory 930 and storage media 920 may be temporary or persistent storage. The program stored in the storage media 920 may include one or more modules, each module including a series of instruction operations on the electronic device. Furthermore, the CPU 910 may be configured to communicate with the storage media 920 and execute a series of instruction operations in the storage media 920 on the electronic device 900. The electronic device 900 may also include one or more power supplies 960, one or more wired or wireless network interfaces 950, one or more input / output interfaces 940, and / or one or more operating systems 921, such as Windows Server. TM Mac OS X TM Unix TM Linux™, FreeBSD™, etc.

[0188] The input / output interface 940 can be used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of the electronic device 900. In one example, the input / output interface 940 includes a network interface controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the input / output interface 940 may be a radio frequency (RF) module for wireless communication with the Internet.

[0189] Those skilled in the art will understand that Figure 12 The structure shown is for illustrative purposes only and does not limit the structure of the electronic device described above. For example, the electronic device 900 may also include... Figure 12 The more or fewer components shown, or having the same Figure 12 The different configurations shown.

[0190] Embodiments of this application also provide a computer-readable storage medium, which can be disposed in an electronic device to store at least one instruction or at least one program related to implementing a feature identification point data processing method in the method embodiment. The at least one instruction or the at least one program is loaded and executed by the processor to implement the feature identification point data processing method provided in the above method embodiment.

[0191] Optionally, in this embodiment, the storage medium may be located at at least one of the multiple network servers in a computer network. Optionally, in this embodiment, the storage medium may include, but is not limited to, various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0192] According to one aspect of this application, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the methods provided in the various alternative implementations described above.

[0193] As can be seen from the embodiments of the feature identification point data processing method, apparatus, device, server, terminal, storage medium, and program product provided in this application, the technical solution of this application receives identification point data of road images reported by the terminal on the server side. The identification point data includes image time, trajectory point data associated with the road image, and identification targets and corresponding feature labels of road features in the road image. The feature labels represent the feature type of the road features. The identification point data is initially aggregated based on time sequence to obtain multiple identification point sequences. In two temporally adjacent identification point sequences, the difference between the starting identification point data of the earlier identification point sequence and the ending identification point data of the later identification point sequence satisfies a preset difference condition. Then, identification point subsequences corresponding to feature labels of at least one feature type are extracted from each of the multiple identification point sequences. The identification target difference and image time difference between two adjacent identification point data in the identification point subsequence are determined. Then, the identification point data in the identification point subsequence are aggregated with the same features according to the identification target difference, image time difference, and feature similarity condition to obtain the target sequence segment corresponding to the identification point sequence. The identification point data in the target sequence segment belong to the same road feature group. In this way, the server receives the identification point data reported by the terminal without needing to receive the overall image information. This reduces traffic and resource consumption while ensuring the coverage and integrity of road elements in the road image, avoiding missing the best identification point. Furthermore, it performs initial grouping by element labels that represent element types, and then further refines the grouping based on differences in identification targets and image time. This enables precise differentiation of data groups of the same road element. The terminal only needs to identify the identification target and element type, without needing to group the same road elements, thus reducing the terminal's data collection, identification, and equipment costs.

[0194] It should be noted that the order of the embodiments described above is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. Furthermore, the above description focuses on specific embodiments of this application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims can be performed in a different order than that shown in the embodiments and still achieve the desired results. Additionally, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired results. In some implementations, multitasking and parallel processing are also possible or may be advantageous.

[0195] The various embodiments in this application are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the device, equipment, and storage medium embodiments are basically similar to the method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0196] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware, or by a program instructing the relevant hardware to implement them. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0197] The above are merely preferred embodiments of this application and are not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for processing feature identification point data, characterized in that, Applied to a server, the method includes: The receiving terminal reports road image recognition point data, which includes image time, trajectory point data associated with the road image, recognition targets of road elements in the road image, and element labels corresponding to the recognition targets. The element labels represent the element type of the road elements. The identification point data is initially aggregated based on time sequence to obtain multiple identification point sequences. In two time-adjacent identification point sequences, the difference between the starting identification point data of the earlier identification point sequence and the ending identification point data of the later identification point sequence meets a preset difference condition. Extract at least one subsequence of identification points corresponding to feature labels of at least one feature type from each of the plurality of identification point sequences; Determine the difference in the target of recognition and the difference in image time between two adjacent recognition points in the recognition point subsequence; Based on the identified target differences, image time differences, and feature similarity conditions, the identified point data in the identified point subsequence are aggregated for the same features to obtain the target sequence segment corresponding to the identified point sequence. The identified point data in the target sequence segment belong to the same road feature group. The identified target differences include the reduction in the number of identified targets and the reduction rate of the identified target area. The reduction rate of the identified target area represents the reduction ratio of the identified target in the later identified point data compared to the identified target in the earlier identified point data among two adjacent identified point data. The feature similarity conditions include at least one of the following sub-conditions: the reduction rate of the identified target area is less than or equal to a ratio threshold; the image time difference is less than or equal to a time difference threshold; and the reduction in the number of identified targets is less than a quantity threshold.

2. The method according to claim 1, characterized in that, The step of aggregating the identification point data in the identification point sub-sequence based on the identification target difference, image time difference, and feature similarity conditions to obtain the target sequence segment corresponding to the identification point sequence includes: If either the difference in the identified target or the difference in the image time does not meet the element similarity condition, it is determined that the data of the two adjacent identification points belong to different road elements. The adjacent two identification points in the identification point subsequence, which belong to different road elements, are grouped and segmented to obtain the target sequence segment corresponding to the identification point sequence.

3. The method according to claim 1, characterized in that, The initial aggregation process of the identification point data based on time sequence yields multiple identification point sequences, including: The identification point data is aggregated based on the time sequence and the preset difference conditions to obtain multiple initial identification point sequences; The abnormal identification points in the initial identification point sequence are filtered out to obtain the identification point sequence.

4. The method according to claim 3, characterized in that, The step of filtering out abnormal identification points from multiple initial identification point sequences to obtain the identification point sequence includes: If, in the initial identification point sequence, the distance between the trajectory points of two consecutive identification point data with the same feature label is within a preset distance range, and the rate of change of the identification target is within a preset rate of change range, the later identification point data among the two identification point data will be identified as an abnormal identification point. Remove the abnormal identification points from the initial identification point sequence.

5. The method according to any one of claims 1-4, characterized in that, The method further includes: Receive the sampling trajectory point data reported by the terminal at interval sampling intervals; The sampled trajectory point data are spliced ​​together based on the time sequence to obtain a set of sampled trajectory sequences; From the sampled trajectory sequences of the sampled trajectory sequence set, determine the target trajectory sequence that matches the target sequence segment; The target sequence segment and the target trajectory sequence are fused based on time sequence to obtain a fused trajectory sequence.

6. The method according to claim 5, characterized in that, The method further includes: Target identification point data is determined from the identification point data in the target sequence segment; The frame timestamp corresponding to the fusion trajectory sequence is determined based on the image time of the target identification point data and the frame point filtering conditions corresponding to the element labels. The frame timestamp is sent to the terminal so that the terminal can filter out road images whose image time matches the frame timestamp.

7. The method according to claim 6, characterized in that, The identification target includes the bounding box coordinates of the road element in the road image; determining the target identification point data from the identification point data in the target sequence segment includes: The area of ​​the recognition box corresponding to each recognition point data in the target sequence segment is determined based on the frame coordinates. The identification point data with the largest identification box area in the target sequence segment is determined as the target identification point data.

8. The method according to claim 5, characterized in that, The step of concatenating the sampled trajectory point data based on time sequence to obtain the sampled trajectory sequence set includes: The sampled trajectory point data is segmented and spliced ​​based on time sequence and preset segmentation interval to obtain an initial sampled trajectory sequence set; Each initial sampling trajectory sequence in the initial sampling trajectory sequence set is deduplicated to obtain the sampling trajectory sequence set.

9. A feature identification point data processing device, applied to a server, characterized in that, The device includes: Identification point data receiving module: used to receive identification point data of road images reported by the terminal. The identification point data includes image time, trajectory point data associated with the road image, identification targets of road elements in the road image and element labels corresponding to the identification targets. The element labels represent the element type of the road elements. First aggregation module: used to perform initial aggregation processing on the identification point data based on time sequence to obtain multiple identification point sequences. In two time-adjacent identification point sequences, the difference between the starting identification point data of the earlier identification point sequence and the ending identification point data of the later identification point sequence meets a preset difference condition. Identification point sequence extraction module: used to extract identification point sequences corresponding to feature labels of at least one feature type from each of the plurality of identification point sequences; Difference determination module: used to determine the difference in recognition target and the difference in image time between two adjacent recognition point data in the recognition point sub-sequence; The second aggregation module is used to aggregate the identification point data in the identification point sub-sequence according to the identification target difference, image time difference, and feature similarity conditions, to obtain the target sequence segment corresponding to the identification point sub-sequence. The identification point data in the target sequence segment belong to the same road feature group. The identification target difference includes the reduction in the number of identification targets and the reduction rate of the identification target area. The reduction rate of the identification target area represents the reduction ratio of the identification target in the later identification point data compared to the identification target in the earlier identification point data among two adjacent identification point data. The feature similarity conditions include at least one of the following sub-conditions: the reduction rate of the identification target area is less than or equal to a ratio threshold; the image time difference is less than or equal to a time difference threshold; and the reduction in the number of identification targets is less than a quantity threshold.

10. The apparatus according to claim 9, characterized in that, The second aggregation module includes: If either the difference in the identified target or the difference in the image time does not meet the element similarity condition, it is determined that the data of the two adjacent identification points belong to different road elements. The adjacent two identification points in the identification point subsequence, which belong to different road elements, are grouped and segmented to obtain the target sequence segment corresponding to the identification point sequence.

11. The apparatus according to claim 9, characterized in that, The first aggregation module includes: Data aggregation unit: used to aggregate the identification point data based on time sequence and the preset difference conditions to obtain multiple initial identification point sequences; Identification point filtering unit: used to filter out abnormal identification points in multiple initial identification point sequences to obtain the identification point sequence.

12. The apparatus according to claim 11, characterized in that, The identification point filtering unit includes: Anomaly point determination subunit: If the distance between the trajectory points of two consecutive identification point data with the same feature label in the initial identification point sequence is within a preset distance range, and the rate of change of the identification target is within a preset rate of change range, the later identification point data in the two identification point data is identified as an anomaly identification point; Anomaly removal subunit: used to remove the anomaly identification points from the initial identification point sequence.

13. The apparatus according to any one of claims 9-12, characterized in that, The device further includes: Track point receiving module: used to receive the sampled track point data of the terminal reported by the interval sampling; Trajectory stitching module: used to stitch together the sampled trajectory point data based on time sequence to obtain a set of sampled trajectory sequences; Target trajectory sequence determination module: used to determine the target trajectory sequence that matches the target sequence segment from the sampled trajectory sequences of the sampled trajectory sequence set; Fusion processing module: used to fuse the target sequence segment and the target trajectory sequence based on time sequence to obtain a fused trajectory sequence.

14. The apparatus according to claim 13, characterized in that, The device further includes: Target data determination module: used to determine target identification point data from the identification point data in the target sequence segment; Frame capture time determination module: used to determine the frame capture timestamp corresponding to the fusion trajectory sequence based on the image time of the target recognition point data and the frame capture point filtering conditions corresponding to the element labels; Data sending module: used to send the frame timestamp to the terminal so that the terminal can filter out road images whose image time matches the frame timestamp.

15. The apparatus according to claim 14, characterized in that, The identification target includes the bounding box coordinates of the road element in the road image; the target data determination module includes: Frame area determination unit: used to determine the area of ​​the recognition frame corresponding to each recognition point data in the target sequence segment based on the frame coordinates; Identification point data determination unit: used to determine the identification point data with the largest identification box area in the target sequence segment as the target identification point data.

16. The apparatus according to claim 13, characterized in that, The trajectory stitching module includes: Trajectory segmentation and splicing unit: used to segment and splice the sampled trajectory point data based on time sequence and preset segmentation interval to obtain an initial sampled trajectory sequence set; Trajectory deduplication unit: used to deduplicate each initial sampled trajectory sequence in the initial sampled trajectory sequence set to obtain the sampled trajectory sequence set.

17. A computer-readable storage medium, characterized in that, The storage medium stores at least one instruction or at least one program segment, which is loaded and executed by a processor to implement the feature identification point data processing method as described in any one of claims 1-8.

18. A computer device, characterized in that, The device includes a processor and a memory, the memory storing at least one instruction or at least one program, the at least one instruction or the at least one program being loaded and executed by the processor to implement the feature identification point data processing method as described in any one of claims 1-8.

19. A computer program product, characterized in that, The computer program product includes computer instructions that, when executed by a processor, implement the feature identification point data processing method as described in any one of claims 1-8.

Images