Data processing method and device, storage medium, electronic equipment and vehicle

Abstract

The invention discloses a data processing method and device, a storage medium, electronic equipment and a vehicle, and relates to the technical field of image processing, and the data processing method comprises the steps: obtaining to-be-visualized data, and carrying out the type conversion of the to-be-visualized data, and obtaining a data set comprising point cloud data and / or mark data; based on the physical attributes of the data in the data set, calculating the overall mass point position of the data set as a target position; and adjusting the central position of the view to a target position so as to display the to-be-visualized data. According to the method, the overall mass point position is calculated based on the physical attributes of the data in the data set, and the central position of the visual view is automatically adjusted accordingly, so that efficient focusing display and interactive prompt of the multi-source heterogeneous to-be-visualized data in the intelligent driving scene are realized.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a data processing method, device, storage medium, electronic equipment and vehicle. Background Art

[0002] When processing vehicle sensor data, the intelligent driving domain controller can visualize the collected vehicle sensor data in various ways to perform data analysis and various processing.

[0003] When visualizing such data, the currently used visualization tools can display the data to be visualized in the view according to its own coordinate points, and the initial view is the default zero point and the default zoom ratio.

[0004] If the real data is concentrated around the zero point, the display method at this time can meet the user's needs. In actual situations, the data to be visualized may not necessarily be concentrated around the zero point. At this time, the display method cannot meet the user's needs. The user needs to manually adjust the view center point and zoom ratio. If the data is far away from the zero point, it is difficult to manually adjust the view, which is inconvenient for users to observe and analyze the data. Summary of the Invention

[0005] The present invention aims to solve at least one of the technical problems in the related art to a certain extent. To this end, one object of the present invention is to provide a data processing method, device, storage medium, electronic device and vehicle to improve the convenience of adjusting and observing views.

[0006] According to a first aspect of an embodiment of the present invention, a data processing method is provided, the method comprising:

[0007] Acquire data to be visualized, and perform type conversion on the data to be visualized to obtain a data set including point cloud data and / or labeled data;

[0008] Calculating the particle position of the entire data set as the target position based on the physical properties of the data in the data set;

[0009] The center position of the view is adjusted to the target position for displaying the data to be visualized.

[0010] According to a second aspect of an embodiment of the present invention, there is provided a data processing device, the device comprising:

[0011] A data acquisition module is used to acquire data to be visualized and perform type conversion on the data to be visualized to obtain a data set including point cloud data and / or labeled data;

[0012] a particle calculation module, configured to calculate the particle positions of the entire data set as target positions based on the physical properties of the data in the data set;

[0013] The view adjustment module is used to adjust the center position of the view to the target position for displaying the data to be visualized.

[0014] According to a third aspect of an embodiment of the present invention, a data processing system is provided. The data processing system is applied to a vehicle, and the system includes:

[0015] At least one sensor, configured to collect data to be visualized, wherein the sensor includes a camera, a laser radar, or a millimeter-wave radar;

[0016] a controller, communicatively connected to the at least one sensor, configured to receive the data to be visualized, perform type conversion on the data to be visualized, and obtain a data set including point cloud data and / or labeled data; calculate, based on physical properties of data in the data set, a particle position of the entire data set as a target position; and adjust a center position of a view to the target position;

[0017] A display device is communicatively connected to the controller, and is used to display the data to be visualized in a display interface with an adjusted view.

[0018] According to a fourth aspect of an embodiment of the present invention, a vehicle is provided, comprising the data processing system according to the third aspect.

[0019] According to a fifth aspect of an embodiment of the present invention, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the above-mentioned data processing method is implemented.

[0020] According to a sixth aspect of an embodiment of the present invention, an electronic device is provided, comprising: a memory and a processor; a computer program is stored in the memory, and when the computer program is executed by the processor, the above-mentioned data processing method is implemented.

[0021] In the solution provided by the embodiments of the present invention, by acquiring the data to be visualized and performing type conversion on the data to be visualized to convert it into a data set including two major types of data: point cloud data and / or labeled data, sensor data with diverse sources and heterogeneous structures can be unified into a unified processing format, which is conducive to compatible processing of multiple types of input data, reduces the complexity of the data structure in the visualization processing flow, and improves the versatility and scalability of the data processing module. Based on the physical properties of the data in the data set, the particle positions of the entire data set are calculated, which can accurately estimate the concentrated area of ​​the current data set in three-dimensional space, effectively improve the spatial perception accuracy of the data visualization process, and provide a reliable basis for subsequent view focusing and intelligent display. Furthermore, the center position of the view is adjusted to the target position for displaying the data to be visualized. By aligning the center of the view with the particle position of the data set, the visualization display can automatically focus on the most representative area of ​​the data, improving the information density and readability of the display interface, avoiding the operational burden caused by manual dragging and zooming of the view by the user, and facilitating user observation and further interaction, thereby helping to improve the overall visualization experience and interaction efficiency.

[0022] Additional aspects and advantages of the present invention will be set forth in part in the description which follows and, in part, will be obvious from the description which follows, or may be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 This is a flow chart of a first data processing method provided by an embodiment of the present invention;

[0024] Figure 2 is a flow chart of a second data processing method provided by an embodiment of the present invention;

[0025] Figure 3 is a flowchart of a third data processing method provided by an embodiment of the present invention;

[0026] Figure 4 This is a schematic diagram of a scenario provided by an embodiment of the present invention;

[0027] Figure 5 is a flowchart of a fourth data processing method provided by an embodiment of the present invention;

[0028] Figure 6 is a structural diagram of a data processing device provided by an embodiment of the present invention;

[0029] Figure 7 It is a structural diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0030] The following describes embodiments of the present invention in detail, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and are not to be construed as limiting the present invention.

[0031] The data processing method, apparatus, storage medium, and electronic device according to embodiments of the present invention are described below with reference to the accompanying drawings.

[0032] In one embodiment of the present invention, see Figure 1 , provides a data processing method, which includes the following steps S101-S103.

[0033] S101: Acquire data to be visualized, and perform type conversion on the data to be visualized to obtain a data set including point cloud data and / or labeled data;

[0034] S102: Calculating the particle position of the entire data set as the target position based on the physical properties of the data in the data set;

[0035] S103: Adjusting the center position of the view to the target position for displaying the data to be visualized.

[0036] Data to be visualized refers to raw input data that has not yet been visualized but can be displayed on a three-dimensional or two-dimensional interface. It is usually output in real time by sensors or upper-level perception modules carried by the vehicle, and includes one or more spatial data and semantic data used to describe the vehicle's surrounding environment.

[0037] In the present invention, the data to be visualized includes but is not limited to the following two categories: three-dimensional sampling data from sensors, such as point cloud data and depth image data from lidar, millimeter-wave radar or cameras; and structured object data from the environmental perception module, such as recognition results of obstacles, parking space frames, target vehicles, etc., which usually contain semantic information such as target category, boundary contour, and center of mass position.

[0038] After receiving the data to be visualized, it is necessary to identify its type and convert its format to standardize it into a data set in a unified format (including point cloud data and / or labeled data) for subsequent particle position calculation and display view adjustment.

[0039] Among them, point cloud data refers to a set of data collected by sensors (such as lidar, millimeter wave radar or camera, etc.) to represent the position of discrete points in space, which is expressed as a set of sampling points in a three-dimensional area. Each point cloud data point usually includes three-dimensional spatial coordinate information (x, y, z). In an embodiment of the present invention, point cloud data is used to describe the spatial distribution characteristics of objects or areas in the environment around the vehicle, and each data point is regarded as an independent entity and has a uniformly set default quality value. For example, a road in an actual scene can be characterized by point cloud data, and the position of each point is at the edge or inside of the road.

[0040] Marker data refers to the annotation information generated by high-level perception algorithms (such as obstacle detection, lane recognition, parking space recognition, dynamic target tracking, etc.) for the structured description of a specific target or area. Each piece of marker data corresponds to a structured object with clear boundaries, category attributes and spatial positions, such as an obstacle, parking space frame, etc. In an embodiment of the present invention, the marker data uses a structured object as the minimum processing unit, that is, each structured object is a monomer, and each monomer has its center of mass position and mass attributes, where the mass can be obtained by calculating the spatial volume and preset density of the structured object. The marker data appears in the three-dimensional view area in the form of points, lines, surfaces, bodies, etc.

[0041] For all types of real vehicle data collected, after receiving any data to be visualized through various communication methods, the data to be visualized can be processed into one of point cloud data and labeled data, and the format is unified into the Robot Operating System 2.0 (ROS2) format.

[0042] Physical properties refer to the basic parameters used to describe the physical distribution characteristics of each monomer in a data set. They include at least the spatial position and mass of each monomer. Spatial position refers to the coordinates of the monomer in three-dimensional space, usually expressed as (x, y, z). Mass refers to the constant used to calculate the overall mass position of a monomer. For example, it can be set to a default constant (such as 1.0) or calculated by multiplying its volume by a preset density.

[0043] A mass point location is a three-dimensional coordinate representing the spatial distribution of a set of data. It is obtained by calculating the weighted relationship between the spatial position and mass of all entities in the data set. For example, a mass point location is a three-dimensional coordinate determined by centroid calculation based on the spatial position and mass properties of entities in the data set (such as points in a point cloud or labeled objects in labeled data).

[0044] The target position refers to the spatial reference position used to adjust the display of the view when performing visualization processing. In an embodiment of the present invention, the target position is the particle position of the entire data set, which is used to indicate the concentrated area in space where the current data to be visualized is located. The center position of the view will be adjusted according to the target position, so that the visualization interface can focus on the area distributed in the current data set, thereby improving the display effect and ease of operation. The target position can be obtained based on the fusion of single-frame or multi-frame data. For example, when multiple data frames are involved, the average value can be calculated based on the particle position of each frame and used as the global target position.

[0045] The calculation method for mass point positions varies in different scenarios. For example, for scenarios consisting only of point cloud data, mass point positions can be directly averaged across all point coordinates. For labeled data, weighted positions are calculated based on the center of mass position and mass of each labeled object. For mixed scenarios containing both point cloud and labeled data, the mass point positions and total mass of both types of data are calculated separately, and then a weighted fusion process is performed. For specific processing methods, please refer to the following examples.

[0046] When calculating the particle position, all data in the data set can be used to calculate the particle position, and a target position can be determined based on the calculated particle position. Then, the center position of the view is adjusted to the target position for displaying the data to be visualized. In this way, by automatically adjusting the view, rather than simply displaying the visualized data on the view based on the position of the data, the view position only adopts the default initial position or the view position adjusted by the last user operation.

[0047] Moreover, compared to the method of extracting part of the data of the main view for the data to be visualized and performing calculations and adjusting the view around the data in the main view, the solution provided by the embodiment of the present invention can take into account all the data to be visualized, so as to achieve the maximum display of data integrity and centralization.

[0048] The view is used to display the data to be visualized in a three-dimensional graphical form. The view can be provided by a preset visualization tool, such as the Mobile Robot Programming Toolkit (mrpt2) tool.

[0049] In some embodiments, adjusting the center position of the view to the target position includes: smoothly adjusting the view of the actual data to be visualized according to the target position. Smooth adjustment refers to the process of gradually approaching the center position of the view to the target position in a gradual manner at a set time interval or movement speed, while avoiding sudden jumps when updating the center position of the view.

[0050] Smooth adjustment makes the current view center move toward the target position with uniform movement speed. The specific implementation method can be:

[0051] At a fixed first time interval, the distance between the center position of the view and the target position is shortened equidistantly until the center position coincides with the target position.

[0052] For example, set a timer that runs once every 100ms, and move the current view center position 1 / 30 toward the target position each time. Finally, the movement is completed in the 3rd second, and the view center position is moved to the target position.

[0053] For example, when adjusting from the view center position, which is also the 3D coordinate zero point of the view (0, 0, 0), to the target position (99, 66, 33), the adjustment is not completed all at once. Instead, a smooth adjustment method is used, where 1 / 30 of the position to be adjusted is adjusted every 100 milliseconds. Finally, the data display view is completed in 3 seconds. Examples of the view positions of the main nodes are shown in Table 1.

[0054] Table 1

[0055] 0s 1s 2s 3s Location (0,0,0) (33,22,11) (66,44,22) (99,66,33)

[0056] The first row in Table 1 indicates the number of seconds that have passed since the start of smooth adjustment, and the second row indicates the coordinates of the center position of the view at different times.

[0057] The center position of the view is initially the default zero position. In addition, if a smooth adjustment has been performed, the center position of the view can be the target position after the last smooth adjustment.

[0058] In this way, by comparing the calculated target position with the current center position of the view, the view position can be automatically adjusted gradually within a reasonable time. Figure 1 It is adjusted to the mass point position in one step, making the user experience smoother and more comfortable.

[0059] Therefore, by adjusting the view position in a smooth adjustment manner, a smooth use effect can be achieved, thereby improving the convenience of adjusting and observing the view, and being able to solve the problem that when displaying various types of data in the 3D area in the visualization tool, the visual effect affects the observation due to view problems.

[0060] In one embodiment of the present invention, after executing step S103, the process may return to the step of obtaining the data to be visualized after a preset second time interval has elapsed. Specifically, the second time interval may be set as needed, and the target position may be recalculated according to steps S101-S103 to adjust the view to accommodate real-time changes in the data's position.

[0061] As can be seen from the above, the solution provided by the embodiment of the present invention obtains the data to be visualized and performs type conversion on the data to be visualized to convert it into a data set including two major types of data: point cloud data and / or labeled data. This can unify sensor data with diverse sources and heterogeneous structures into a unified processing format, facilitate compatibility processing of multiple types of input data, reduce the complexity of the data structure in the visualization processing flow, and improve the versatility and scalability of the data processing module. Based on the physical properties of the data in the data set, the particle position of the entire data set is calculated, which can accurately estimate the concentration area of ​​the current data set in three-dimensional space, effectively improve the spatial perception accuracy in the data visualization process, and provide a reliable basis for subsequent view focusing and intelligent display. Furthermore, the center position of the view is adjusted to the target position for displaying the data to be visualized. By aligning the center of the view with the particle position of the data set, the visualization display can automatically focus on the most representative area of ​​the data, improving the information density and readability of the display interface, avoiding the operational burden caused by manual dragging and zooming of the view by the user, and facilitating user observation and further interaction, thereby helping to improve the overall visualization experience and interaction efficiency.

[0062] In addition, in some embodiments, the type conversion of the data to be visualized to obtain a data set including point cloud data and / or labeled data includes: identifying the data source of the data to be visualized; converting the three-dimensional sampling point data from the sensor into point cloud data, and / or converting the structured object data from environmental perception into labeled data; the structured object data includes at least obstacle information and parking space frame information.

[0063] Structured object data derived from environmental perception refers to data in a structured description format that is output by the vehicle's environmental perception system (e.g., target detection modules based on cameras, radars, and ultrasonic sensors) after identifying and semantically analyzing the surrounding environment. Structured object data is typically used to describe static or dynamic targets in a scene. Its data structure includes, but is not limited to, target objects (e.g., obstacles, parking spaces, traffic signs, pedestrians, etc.), spatial positioning information (e.g., geometric boundaries, bounding boxes, center of mass coordinates), spatial dimension information (e.g., length, width, height, contour points), and optional semantic tags.

[0064] The environmental perception system refers to the combination of a sensor group and its processing unit installed on the vehicle for sensing the surrounding environment. The target detection module is a functional unit within the environmental perception system, responsible for identifying and locating semantically labeled target objects, such as vehicles, pedestrians, obstacles, lane markings, and parking spaces, from the raw sensory data. This module typically uses computer vision or deep learning algorithms to extract target boundary information (such as 2D and 3D boxes, categories, or confidence levels) from image frames, radar reflection points, or point cloud data, and outputs the recognition results in a structured format.

[0065] For example, while a vehicle is driving, the environmental perception system detects two obstacles and an empty parking space in front of the vehicle. It then describes these three objects as structured data objects, including their center points, boundary outlines, spatial dimensions, and category labels. This type of structured description is the structured object data derived from environmental perception.

[0066] Therefore, by unifying multi-source heterogeneous data into a standardized representation, the data processing compatibility of the visualization system can be significantly improved, providing data support for subsequent particle calculations.

[0067] In some embodiments, the above method is applied to an intelligent driving domain controller of a vehicle; the vehicle is equipped with multiple sensors for collecting data to be visualized; the above method also includes: in response to a manual triggering operation of a user or an automatic triggering operation of a vehicle, displaying the data to be visualized in an adjusted view in a display interface to assist in determining at least one of driving strategies, target object identification confirmation, or human-computer interaction prompts.

[0068] Among them, the display interface refers to the graphical user interface (GUI) in the vehicle used to present visual data content. The interface can be deployed in the vehicle's central control screen, instrument display, head-up display (HUD) or other display devices with image display capabilities.

[0069] The user's manual triggering operation includes but is not limited to one or more of touch operation, key operation, voice instruction, etc.

[0070] Automatically triggered vehicle actions can be based on sensor detection results, driving status, or algorithmic logic. For example, in intelligent driving scenarios, when a new obstacle is detected ahead of the vehicle, the route changes, or the driving strategy is updated, the intelligent driving domain controller automatically acquires the data set, merges it, calculates the target position, and adjusts the center position of the view to the target position for display, without requiring user intervention.

[0071] When a user manually operates the human-machine interface, or the vehicle automatically triggers a display action under specific conditions, the intelligent driving domain controller updates the display interface based on the adjusted target position, allowing the user or system to focus on a specific spatial area. This display interface can be used to assist in driving strategy formulation, target recognition and confirmation, or provide real-time interactive prompts.

[0072] Among them, driving strategy refers to the decision-making plan made by the vehicle to achieve safe, stable and efficient driving behavior under a given driving task or driving goal.

[0073] Target object recognition and confirmation refers to the process of explicitly displaying and manually or automatically verifying environmental targets (such as vehicles, pedestrians, obstacles, lane lines, etc.) identified by the perception system.

[0074] Human-computer interaction prompts refer to the interactive means by which the system conveys the current environmental perception status or system operation suggestions to the user through graphics, text, icons, color changes or sounds on the display interface, such as highlighting the focus area, prompting vehicle deviation, warning of target approaching, etc.

[0075] For example, users can also mark the currently displayed visual data area as a focus area for subsequent analysis and processing or event tracing. Alternatively, the intelligent driving domain controller can also upload the image information of the display interface to a remote platform for logging, remote monitoring, or anomaly analysis.

[0076] The following example illustrates how to calculate the particle position and determine the target position.

[0077] In a first implementation manner, when the data set includes only point cloud data, the particle position of the point cloud data can be obtained according to the position of the point represented by the point cloud data as the target position.

[0078] In this implementation, each item of data contained in the point cloud data in the data set is a point in the point cloud. The position of the point is represented by three-dimensional coordinates. To simplify the calculation process, a unified default value can be used for the volume and density of each structured object in the labeled data. Both the volume and density are set to default values. For example, if the calculation accuracy requirements allow, the volume of all labeled objects can be set to a default constant value, and the default value can be 1.0 unit volume; at the same time, the density parameter can also be set to a unified default value, such as 1.0 unit mass / volume. For the specific calculation method, please refer to the formula in the second implementation method below.

[0079] In the second implementation method, when the data set only includes labeled data, the particle position of each monomer represented by the labeled data can be calculated; the particle position of the labeled data is determined based on the particle position of each monomer and the mass of each monomer, and the particle position is used as the target position.

[0080] Due to the presence of different obstacles, different cells have different volumes. To facilitate calculations, the cell density can be set to the default value of 1 unit. This way, the cell is a uniformly distributed object. For a uniformly distributed object, it can be decomposed into multiple small pieces, and the center of mass of each small piece is calculated. Finally, the weighted sum of the small pieces' volume distribution proportions on the cell is used to obtain the particle position of the cell.

[0081] Specifically, the particle position of the marker data can be calculated as follows:

[0082] The mass of each monomer is weighted and summed using the coordinates corresponding to the mass position of each monomer as the weight to obtain the sum result; the mass position of the marked data is calculated based on the proportion of the sum result in the total mass of all monomers.

[0083] The calculation process is as follows. The numerator on the right side of the formula is the sum of the weighted summation.

[0084]

[0085]

[0086]

[0087] in, are the coordinates of the particle position of the labeled data in the x-axis, y-axis, and z-axis directions; n represents the total number of monomers; x i 、y i 、z i represents the coordinates of the particle position of the i-th monomer in the x, y, and z directions; m i is the mass of the ith monomer.

[0088] Similar to the above formula, in the first implementation, since the point volume and density in the point cloud data are both set to the default value of 1 unit, the formula for calculating the particle position in the first implementation can be simplified as follows:

[0089]

[0090]

[0091]

[0092] in, are the coordinates of the particle position of the point cloud data in the x-axis, y-axis, and z-axis directions, respectively. n represents the number of points in the point cloud data; i 、y i 、z i Indicates the coordinates of the position of the i-th point in the x, y, and z directions; m i is the mass of the i-th point. According to the mass formula m=ρV, mass is equal to density multiplied by volume. For any i, m i Both are 1, so it can be simplified to the expression on the right side of the equation in the formula. There is no need to calculate the mass, and the particle position of the point cloud data can be obtained only by the position of the point.

[0093] According to the above calculation method, taking the data volume ratio as the core and considering the information and ratio of all data can more reasonably highlight the data centrality.

[0094] In a third implementation, when the data set includes point cloud data and labeled data, the mass position of the point cloud data can be obtained based on the position of the point represented by the point cloud data, and the quality of the point cloud data can be determined;

[0095] Calculate the mass point position of each monomer represented by the labeled data;

[0096] The target position is determined based on the mass position and mass of each monomer and the mass position and mass of the point cloud data.

[0097] This implementation is equivalent to a combination of the first and second implementations. The first implementation calculates the mass positions of the point cloud data, and then treats the point cloud as a single entity and applies it to the second implementation for calculations. Because the point volume and density in the point cloud data are both set to the default value of 1 unit, the quality of the point cloud data can be expressed as the number of points.

[0098] In one embodiment, if there are multiple frames of data to be visualized, the target position can be determined as follows:

[0099] Calculating the particle positions included in the data set corresponding to each frame of data to be visualized in the multiple frames of data to be visualized, and obtaining the first target position of each frame of data to be visualized through the particle positions of each frame of data to be visualized;

[0100] The average position of the first target position corresponding to each frame of the data to be visualized in the multiple frames of the data to be visualized is calculated, and the average position is used as the target position.

[0101] The first target position is calculated in the same manner as the target position calculated in the three implementations for calculating the particle position and determining the target position in the aforementioned embodiments, differing only in the name and concept. Specifically, three frames of data transmitted by the sensor are acquired at different time points. The particle position in each frame is calculated using one of the first, second, and third implementations described above. The x, y, and z coordinate values ​​of the particle position across the three frames are then averaged to obtain the average position.

[0102] In some other embodiments, more than 4 frames, 5 frames, or other data to be visualized may be selected, and the embodiment of the present invention is not limited thereto.

[0103] Figure 2 The embodiment shown is a simplified flow of the data processing method provided by the embodiment of the present invention.

[0104] The acquired data is labeled data and / or point cloud data. Three-frame mass points are calculated, that is, the mass point positions are obtained in the three-frame data set, and the final average mass point is calculated, which is the average position. The average position is used to represent the overall mass point of the three frames of data. Then, the view is adjusted to the mass point through smoothing, that is, the center position of the view is moved to the average position.

[0105] Figure 3 The illustrated embodiment illustrates a specific process for a data processing method, wherein the data to be visualized is acquired through various communication methods, including point cloud data and labeled data; and the mass points of three frames of data, i.e., the first target position, are calculated using mass point calculation formulas in different scenarios.

[0106] The scene diagram is as follows: Figure 4 As shown, the data set in scene 1 includes only point cloud data, the data set in scene 2 includes only labeled data, and the data set in scene 3 includes point cloud data and labeled data. Refer to the three implementation methods of calculating the particle position and determining the target position in the aforementioned embodiment, which will not be described in detail here.

[0107] The average value of the three calculated mass points is the final mass point position, which is the target position. The view position can then be adjusted to the final mass point through a smooth adjustment method.

[0108] The detailed steps of the data processing method are as follows Figure 5 shown.

[0109] Among them, multiple channels of various sensor data are obtained, including various types of simply processed sensor data published by the domain control service through the Distributed Data Service (DDS) communication protocol or the Transport for Real-time Operation Systems (TROS) protocol, as the data to be processed.

[0110] After acquisition, the sensor data is unified into a unified format. Since the received sensor data types may have different formats depending on the sensor, the various sensor data are converted into labeled or point cloud data according to classification and unified into a ROS2-based format for processing in subsequent processes.

[0111] After completing the classification conversion, read a frame of data and perform scene classification. The specific scene of the classification is as follows: Figure 4 The embodiment shown. First, determine whether it contains point cloud data. If not, calculate the data particle position according to scenario 2. If the frame data contains point cloud data, further determine whether it contains marker data. If so, calculate the data particle position according to scenario 3. If not, calculate the data particle position according to scenario 1. Finally, obtain the data particle position of a frame according to the scenario, which is the first target position.

[0112] Furthermore, it is determined whether there are already three values, that is, the first target position of the three frames is calculated. If not, the steps of reading one frame of data and the subsequent steps are executed in a loop until the three first target positions are obtained. The average value of the three values ​​is the final particle position, that is, the target position.

[0113] Calculate the distance s between the current position and the mass point. The mass point here is the mass point in the three-frame data set, represented by the target position. Calculate the distance s in the x, y, and z directions that need to be adjusted.

[0114] Adjust the view timing, including defining the timer to start running so that the timer runs every 100ms;

[0115] Check if 100ms has been reached. If so, adjust the current view to move s / 30 toward the mass point. That is, each time the center of the current view moves 1 / 30 toward the target position, the movement is finally completed in the third second. Check if the view has reached the target position. If so, stop the timer and complete the adjustment.

[0116] Corresponding to the above method embodiment, the solution provided by the embodiment of the present invention also includes a data processing device, see Figure 6 , the device comprises:

[0117] The data acquisition module 601 is used to acquire data to be visualized and perform type conversion on the data to be visualized to obtain a data set including point cloud data and / or labeled data;

[0118] A particle calculation module 602 is configured to calculate the particle positions of the entire data set as target positions based on the physical properties of the data in the data set;

[0119] The view adjustment module 603 is configured to adjust the center position of the view to the target position so as to display the data to be visualized.

[0120] As can be seen from the above, the solution provided by the embodiment of the present invention obtains the data to be visualized and performs type conversion on the data to be visualized to convert it into a data set including two major types of data: point cloud data and / or labeled data. This can unify sensor data with diverse sources and heterogeneous structures into a unified processing format, facilitate compatibility processing of multiple types of input data, reduce the complexity of the data structure in the visualization processing flow, and improve the versatility and scalability of the data processing module. Based on the physical properties of the data in the data set, the particle position of the entire data set is calculated, which can accurately estimate the concentration area of ​​the current data set in three-dimensional space, effectively improve the spatial perception accuracy in the data visualization process, and provide a reliable basis for subsequent view focusing and intelligent display. Furthermore, the center position of the view is adjusted to the target position for displaying the data to be visualized. By aligning the center of the view with the particle position of the data set, the visualization display can automatically focus on the most representative area of ​​the data, improving the information density and readability of the display interface, avoiding the operational burden caused by manual dragging and zooming of the view by the user, and facilitating user observation and further interaction, thereby helping to improve the overall visualization experience and interaction efficiency.

[0121] In one embodiment of the present invention, a data processing system is provided for use in a vehicle. The data processing system includes:

[0122] At least one sensor, configured to collect data to be visualized, wherein the sensor includes a camera, a laser radar, or a millimeter-wave radar;

[0123] an intelligent driving domain controller, communicatively connected to the at least one sensor, configured to receive the data to be visualized, perform type conversion on the data to be visualized, and obtain a data set including point cloud data and / or labeled data; calculate, based on physical properties of the data in the data set, a particle position of the entire data set as a target position; and adjust the center position of the view to the target position;

[0124] A display device is in communication with the controller and is configured to display the data to be visualized in an adjusted view on a display interface. The data to be visualized is used to at least assist in determining a driving strategy and may also be used for object recognition, confirmation, or human-computer interaction prompts.

[0125] Among them, display devices refer to electronic devices with image display capabilities, including but not limited to vehicle-mounted central control screens, instrument display screens, or head-up displays (HUDs).

[0126] Please refer to the above-mentioned embodiment for the specific process, which will not be described again here.

[0127] In one embodiment of the present invention, an intelligent driving domain controller is also provided. The intelligent driving domain controller is integrated in a vehicle and includes a processor and a memory. The memory stores a computer program. When the computer program is executed by the processor, it implements the data processing method as described in any of the above embodiments.

[0128] In one embodiment of the present invention, a vehicle is further provided, which includes the data processing system provided by the above embodiment, or includes the intelligent driving domain controller provided by the above embodiment.

[0129] In one embodiment of the present invention, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the data processing method of any of the above embodiments is implemented.

[0130] In one embodiment of the present invention, an electronic device is provided, comprising: a memory and a processor; a computer program is stored in the memory, and when the computer program is executed by the processor, the data processing method of any of the above embodiments is implemented.

[0131] Figure 7 It is a structural block diagram of an electronic device according to an embodiment of the present invention.

[0132] like Figure 7 As shown, electronic device 700 includes: a processor 701 and a memory 703. Processor 701 and memory 703 are connected, for example, via a bus 702. Optionally, electronic device 700 may further include a transceiver 704. It should be noted that in actual applications, the number of transceivers 704 is not limited to one, and the structure of electronic device 700 does not constitute a limitation on the embodiments of the present invention.

[0133] The processor 701 may be a central processing unit (CPU), a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It may implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the present disclosure. The processor 701 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, and the like.

[0134] The bus 702 may include a path for transmitting information between the above components. The bus 702 may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus 702 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 7 Only one thick line is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0135] The memory 703 is used to store a computer program corresponding to the data processing method of the above embodiment of the present invention, and the computer program is controlled and executed by the processor 701. The processor 701 is used to execute the computer program stored in the memory 703 to implement the content shown in the above method embodiment.

[0136] Among them, the electronic device 700 includes but is not limited to: mobile terminals such as mobile phones, laptops, digital broadcast receivers, personal digital assistants (PDAs), tablet computers (PADs), portable multimedia players (PMPs), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 7 The electronic device 700 shown is merely an example and should not limit the functions and scope of use of the embodiments of the present invention.

[0137] In one embodiment of the present invention, a vehicle is further provided, comprising: a data processing system, a memory, and a controller;

[0138] The memory is used to store executable instructions;

[0139] The controller is electrically connected to the memory and the data processing system respectively, and is used to execute executable instructions stored in the memory to implement the data processing method described in any of the above embodiments in the data processing system.

[0140] It should be noted that the logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing the logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device), or in conjunction with such instruction execution system, apparatus, or device. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transmit a program for use by an instruction execution system, apparatus, or device, or in conjunction with such instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include the following: an electrical connection with one or more wires (electronic devices), a portable computer disk cartridge (magnetic devices), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM) or flash memory, a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering, or processing in another suitable manner as necessary, and then stored in a computer memory.

[0141] It should be understood that various parts of the present invention can be implemented using hardware, software, firmware, or a combination thereof. In the above-described embodiments, multiple steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application-specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0142] Throughout this specification, reference to terms such as "one embodiment," "some embodiments," "examples," "specific examples," or "some examples" means that a specific feature, structure, material, or characteristic described in conjunction with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0143] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise", "axial", "radial", "circumferential" and the like to indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as limiting the present invention.

[0144] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of the technical features being referred to. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of the present invention, "plurality" means at least two, such as two, three, etc., unless otherwise specifically defined.

[0145] In the present invention, unless otherwise specified or limited, the terms "installed," "connected," "connect," "fixed," etc. should be understood in a broad sense. For example, they can refer to fixed connection, detachable connection, or integration; mechanical connection, electrical connection; direct connection, or indirect connection through an intermediate medium; internal communication between two components, or interaction between two components, unless otherwise specified. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on specific circumstances.

[0146] In the present invention, unless otherwise expressly specified or limited, when a first feature is "above" or "below" a second feature, it may mean that the first and second features are in direct contact, or that the first and second features are in indirect contact through an intermediary. Furthermore, when a first feature is "above," "above," or "above" a second feature, it may mean that the first feature is directly above or diagonally above the second feature, or simply means that the first feature is at a higher level than the second feature. When a first feature is "below," "below," or "below" a second feature, it may mean that the first feature is directly below or diagonally below the second feature, or simply means that the first feature is at a lower level than the second feature.

[0147] Although the embodiments of the present invention have been shown and described above, it will be understood that the above embodiments are illustrative and are not to be construed as limitations on the present invention. A person skilled in the art may change, modify, replace and modify the above embodiments within the scope of the present invention.

Claims

1. A data processing method, characterized in that: The method comprises: Acquire data to be visualized, and perform type conversion on the data to be visualized to obtain a data set including point cloud data and / or labeled data; Calculating the particle position of the entire data set as the target position based on the physical properties of the data in the data set; The center position of the view is adjusted to the target position for displaying the data to be visualized.

2. The data processing method according to claim 1, wherein: When the data set includes only the point cloud data, the physical properties of the data include the positions of the points; and calculating the positions of the particles of the entire data set based on the physical properties of the data in the data set to obtain the target position includes: Calculating the particle positions according to the positions of the points represented by the point cloud data to obtain the particle positions of the entire data set; The mass point position of the entire data set is taken as the target position.

3. The data processing method according to claim 1, wherein: When the data set includes only the labeled data, the physical properties of the data include the particle position and mass of each individual particle; and calculating the particle position of the entire data set based on the physical properties of the data in the data set to obtain the target position includes: Calculating the particle position of each monomer represented by the label data; The mass point position of the marking data is determined according to the mass point position of each monomer and the mass of each monomer, and the mass point position is used as the target position.

4. The data processing method according to claim 3, wherein: The particle position of the marker data is calculated as follows: performing weighted summation on the masses of the monomers using the coordinates corresponding to the mass positions of the monomers as weights to obtain a summation result; The mass point position of the marking data is calculated based on the proportion of the summation result in the total mass of all monomers.

5. The data processing method according to claim 1, wherein: When the data set includes the point cloud data and the labeled data, the physical properties of the data include the position and mass of the points, and the position and mass of the particle of the monomer; and calculating the particle position of the entire data set based on the physical properties of the data in the data set to obtain the target position includes: Obtaining mass points of the point cloud data according to positions of points represented by the point cloud data, and determining the quality of the point cloud data; Calculating the particle position of each monomer represented by the label data; A target position is determined based on the mass point positions and masses of the individual monomers and the mass point positions and masses of the point cloud data.

6. The data processing method according to claim 1, wherein: If there are multiple frames of data to be visualized, calculating the particle positions of the data included in the data set and obtaining the target position through the particle positions of the data includes: Calculating the particle positions included in the data set corresponding to each frame of the data to be visualized in the multiple frames of the data to be visualized, and obtaining the first target position of each frame of the data to be visualized through the particle positions of each frame of the data to be visualized; An average position of the first target position corresponding to each frame of the data to be visualized in multiple frames of the data to be visualized is calculated, and the average position is used as the target position.

7. The data processing method according to any one of claims 1 to 6, characterized in that: The adjusting the center position of the view to the target position includes: At a fixed first time interval, the distance between the center position of the view and the target position is shortened equidistantly until the center position coincides with the target position.

8. The data processing method according to any one of claims 1 to 6, characterized in that: The step of converting the data to be visualized to obtain a data set including point cloud data and / or labeled data includes: Identifying a data source of the data to be visualized; The three-dimensional sampling point data from the sensor is converted into point cloud data, and / or the structured object data from the environment perception is converted into label data; the structured object data includes at least obstacle information and parking space frame information.

9. The data processing method according to claim 1, wherein: The method is applied to an intelligent driving domain controller of a vehicle; the vehicle is equipped with multiple sensors for collecting data to be visualized; the method further includes: In response to a manual triggering operation by the user or an automatic triggering operation by the vehicle, the data to be visualized is displayed in an adjusted view in the display interface to assist in determining at least one of driving strategies, target object recognition confirmation, or human-computer interaction prompts.

10. A data processing device, characterized in that: The device comprises: A data acquisition module is used to acquire data to be visualized and perform type conversion on the data to be visualized to obtain a data set including point cloud data and / or labeled data; a particle calculation module, configured to calculate the particle positions of the entire data set as target positions based on the physical properties of the data in the data set; The view adjustment module is used to adjust the center position of the view to the target position for displaying the data to be visualized.

11. A data processing system, characterized in that: The data processing system is applied to a vehicle, and the system includes: At least one sensor, configured to collect data to be visualized, wherein the sensor includes a camera, a laser radar, or a millimeter-wave radar; an intelligent driving domain controller, communicatively connected to the at least one sensor, configured to receive the data to be visualized, perform type conversion on the data to be visualized, and obtain a data set including point cloud data and / or labeled data; calculate, based on physical properties of the data in the data set, a particle position of the entire data set as a target position; and adjust the center position of the view to the target position; A display device is communicatively connected to the controller, and is used to display the data to be visualized in an adjusted view in a display interface; the data to be visualized is at least used to assist in determining a driving strategy.

12. An intelligent driving domain controller, characterized in that: The intelligent driving domain controller includes a processor and a memory, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the data processing method according to any one of claims 1 to 9 is implemented.

13. A vehicle, characterized in that: The vehicle includes the data processing system according to claim 11, or includes the intelligent driving domain controller according to claim 12.

14. An electronic device, characterized in that: The method comprises a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the data processing method according to any one of claims 1 to 9 is implemented.

15. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the data processing method according to any one of claims 1 to 9 is implemented.

Images