Visual user-defined modeling method and device

Abstract

The invention relates to the technical field of automatic driving visual modeling, and discloses a visual self-defined modeling method and device. The method comprises the following steps: acquiring creation operation information, and creating a visual element type in a visual modeling interface corresponding to a project; obtaining configuration operation information of the visual element type, performing attribute configuration on the visual element type based on the configuration operation information, and rendering an attribute configuration result of the visual element in real time in the visual modeling interface; and obtaining dragging operation information for the visual element type, determining corresponding modeling data based on the dragging operation information in the visual modeling interface, and performing preset format conversion and export on the modeling data by adopting the mapping dictionary. According to the method, a new visual element can be created or the configuration information of the existing element can be modified, and the configuration result of the visual element can be updated in real time in the visual modeling interface, so that the learning cost and the use cost of modeling the vehicle middleware by a user are reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of autonomous driving visualization modeling, and in particular to a visualization custom modeling method and device. Background Art

[0002] With the development of autonomous driving technology, the verification of algorithm middleware deployed on vehicles has received widespread attention in the industry. Currently, there are some mature visualization tools for middleware editing on the market.

[0003] However, the related art often requires users to use the visualization tools dedicated to different types of middleware, and when users want to modify the visualization effect using a visualization tool, they need to modify it in a hard-coded form. The above problems may increase the learning cost and usage cost of users when modeling vehicle middleware, and a better solution is needed. Summary of the invention

[0004] In view of this, the present invention provides a visual custom modeling method and device to solve the problem of high learning cost and usage cost when users model vehicle middleware.

[0005] The present invention provides a visual custom modeling method, which includes obtaining creation operation information and creating a visual component type in a visual modeling interface corresponding to a project; the visual component type includes: canvas rule, canvas type, component type, pile type, and connection type;

[0006] Acquire configuration operation information of the visualization element type, perform attribute configuration on the visualization element type based on the configuration operation information, and render the attribute configuration result of the visualization element in real time in the visualization modeling interface;

[0007] Drag operation information for the visualization element type is obtained, corresponding modeling data is determined in the visualization modeling interface based on the drag operation information, and a mapping dictionary is used to convert and export the modeling data into a preset format.

[0008] Another aspect of the present invention provides a visual custom modeling device, the device comprising: a component creation module, used to obtain creation operation information, and create a visual component type in a visual modeling interface corresponding to the project; the visual component type comprises: canvas rule, canvas type, component type, pile type, connection type;

[0009] An attribute configuration module, used to obtain configuration operation information of the visualization element type, perform attribute configuration on the visualization element type based on the configuration operation information, and render the attribute configuration result of the visualization element in real time in the visualization modeling interface;

[0010] The modeling export module is used to obtain the drag operation information for the visualization element type, determine the corresponding modeling data based on the drag operation information in the visualization modeling interface, and use a mapping dictionary to convert and export the modeling data into a preset format.

[0011] On the other hand, the present invention also provides a computer device, which includes: a memory and a processor, the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor implements the above-mentioned visual custom modeling method by executing the computer instructions.

[0012] Another aspect of the present invention further provides a computer-readable storage medium, on which computer instructions are stored, and the computer instructions are used to enable a computer to implement the above-mentioned visual custom modeling method.

[0013] In this process, new visualization components are created or the configuration information of existing components is modified during the modeling process, and the configuration results of the visualization components are updated in real time in the visual modeling interface, so that users can customize the corresponding visualization components for different types of middleware, thereby reducing the learning cost and usage cost of using visualization tools when users are modeling vehicle middleware. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the related technologies, the drawings required for use in the specific embodiments or the related technical descriptions will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0015] Figure 1 It is a flowchart of a visual custom modeling method provided by an embodiment of the present invention;

[0016] Figure 2 It is a schematic diagram of a flow chart of importing historical engineering files of a visual custom modeling method provided by an embodiment of the present invention;

[0017] Figure 3 It is a schematic diagram of the association of a visual custom modeling method provided by an embodiment of the present invention;

[0018] Figure 4 It is a schematic diagram of an export process of a visual custom modeling method provided by an embodiment of the present invention;

[0019] Figure 5 It is an overall architecture diagram of a visual custom modeling method provided by an embodiment of the present invention;

[0020] Figure 6 is a structural schematic diagram of a visual custom modeling device provided by an embodiment of the present invention;

[0021] Figure 7 It is a structural schematic diagram of another visual custom modeling device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0022] In the field of autonomous driving, algorithm middleware is one of the core technologies for implementing various functions and services. These middleware are usually responsible for managing key tasks such as data processing from different sensors, path planning, decision making, and control execution. With the continuous advancement of technology, several well-received middleware designed specifically for autonomous driving have emerged in the market.

[0023] These middleware interact with various parts of the vehicle through the domain controller to achieve real-time monitoring and management of the vehicle status. For example, sensor data collection and fusion algorithms can quickly identify obstacles in the environment and help the vehicle respond immediately. In addition, the decision-making algorithm can formulate a safe driving path based on real-time data and preset rules to ensure the safety of the vehicle in complex traffic environments.

[0024] At the same time, autonomous driving technology is still in a stage of rapid development, and the algorithms and models involved are becoming increasingly complex. This requires strict verification and testing of middleware to ensure reliability and safety under real driving conditions.

[0025] However, existing visualization tools used for middleware verification and testing often have the following problems: users are required to use different types of visualization tools dedicated to different types of vehicle middleware, and when users want to modify the visualization effect using a certain visualization tool, they need to modify it in a hard-coded form, which increases the learning cost and usage cost of users when using visualization tools when modeling vehicle middleware.

[0026] In order to solve the above problems, various embodiments of the present invention provide a visual custom modeling method, which is characterized in that the method includes: obtaining creation operation information, creating a visual component type in a visual modeling interface corresponding to the project; the visual component type includes: canvas rules, canvas types, component types, pile types, and connection types; obtaining configuration operation information of the visual component type, configuring the properties of the visual component type based on the configuration operation information, and rendering the property configuration results of the visual component in real time in the visual modeling interface; obtaining drag operation information for the visual component type, determining corresponding modeling data in the visual modeling interface based on the drag operation information, and using a mapping dictionary to convert and export the modeling data into a preset format.

[0027] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0028] According to an embodiment of the present invention, a visual custom modeling method is provided. Figure 1 is a flow chart of a visual custom modeling method provided by an embodiment of the present invention, such as Figure 1 As shown, the process includes the following steps:

[0029] Step S101, obtaining creation operation information, and creating a visualization component type in the visualization modeling interface corresponding to the project; the visualization component type includes: canvas rule, canvas type, component type, pile type, and connection type;

[0030] Step S102, obtaining configuration operation information of the visualization component type, performing attribute configuration on the visualization component type based on the configuration operation information, and rendering the attribute configuration result of the visualization component in real time in the visualization modeling interface;

[0031] Step S103, obtaining drag operation information for the visualization component type, determining corresponding modeling data based on the drag operation information in the visualization modeling interface, and converting and exporting the modeling data into a preset format using a mapping dictionary.

[0032] In one possible implementation, canvas rules may refer to logical specifications that users follow when operating and configuring components on the canvas; for example, canvas rules may include but are not limited to: components cannot overlap, lines cannot block each other, the number of links on a component is limited, etc.

[0033] Exemplarily, the canvas rules may specifically refer to the modeling rules of autonomous driving, wherein the canvas rules may include but are not limited to: data flow direction, component connection method; for example, the data flow direction may include at least one of the following: the data of the lidar and the camera flows to the perception module, the environmental information output by the perception module flows to the decision module, and the control instructions generated by the decision module flow to the vehicle speed control and steering system; the component connection method may include at least one of the following: the lidar, the camera and the ultrasonic sensor are connected in parallel to the environmental perception module, and the control module is connected to the actuator through the controller area network (CAN) interface.

[0034] Furthermore, the canvas type can refer to the area or layout for displaying and organizing various visual components based on canvas rules; the component available refers to the basic unit in the modeling tool, which is used to represent an independent function or module in the system; the stub can refer to the interface or node connecting the components, which is used to realize data exchange and communication between components; the line is the visual element connecting different components, representing the path of data flow or control flow.

[0035] Exemplarily, canvas types may include but are not limited to: sensor layout, decision algorithm diagram, etc.; components may include but are not limited to: lidar, camera, vehicle controller, etc.; piles may include but are not limited to: external interface, data interaction lighting; connections may include but are not limited to: ordered transmission mode, parallel transmission mode.

[0036] Furthermore, the visual modeling interface may include a new type or rule area, a canvas rendering area, and a property configuration area; the user creates a visual component type in the new type or rule area, specifically configures the properties of the visual component in the property configuration area, and visually sees the created visual component in the canvas rendering area.

[0037] In a possible implementation, configuring properties of the visualization element type based on the configuration operation information includes:

[0038] Based on the configuration operation information, the properties of the visualization element type are configured to determine the basic configuration, appearance configuration and registration events of the visualization element type.

[0039] Here, the basic configuration may include but is not limited to type, functional parameter layout information, the appearance configuration may include color style, shape, size, and the registered event may include event type and event processing function; wherein, the registered event refers to defining an event response mechanism related to user interaction for a visual element.

[0040] In a possible implementation, a mapping dictionary may refer to a structure that converts input data into a target format according to certain rules through predefined key-value pairs.

[0041] Furthermore, obtaining the user's drag operation information for the visualization element type, determining the corresponding modeling data based on the drag operation information in the visualization modeling interface, and using the mapping dictionary to convert and export the modeling data into a preset format can be achieved based on the following steps:

[0042] Monitor the user's dragging behavior on the visual modeling interface in real time, and obtain the starting position, end position and type of visual element being operated;

[0043] Based on the position where the user ends the drag, the final position of the component is updated to ensure that it is fixed at the target coordinates in the interface, and the user's dragging operation is converted into corresponding modeling data, including the position, size, attributes, etc. of the component;

[0044] The generated modeling data is converted through a mapping dictionary to obtain export data in a preset format, and the converted export data is packaged into a required export format.

[0045] Exemplarily, the export format may include, but is not limited to: JavaScript Object Notation (JSON) format, Extensible Markup Language (XML) format, or a format required by a debugging engine.

[0046] Here, the debugging engine may refer to a software tool used to analyze, detect and troubleshoot errors or problems in modeling data; conversely, the above-mentioned visual modeling system for generating modeling data may refer to a modeling engine.

[0047] Through the above method, the user only needs to create and configure the visualization component type through operations on the interface, such as clicking, dragging, etc., to clearly display the interaction methods of various sensors, control units and actuators in the vehicle middleware, thereby reducing the complexity of using visualization tools and enabling users to quickly and intuitively complete the modeling of the vehicle middleware; through the real-time rendering function of step S102, the configuration effect is instantly displayed according to the user's attribute configuration information, and the user can immediately see the result while operating, which greatly improves the user's interactive experience, allowing the user to quickly adjust the model configuration according to the feedback data in real time, optimize the decision logic or control strategy, thereby avoiding repeated adjustments and waiting time for rendering; through the automatic conversion of mapping dictionaries and preset formats, the system not only simplifies the data export process, but also ensures the compatibility and consistency of the data.

[0048] In a specific embodiment, the method further comprises:

[0049] Acquire project creation information, generate a project file of the project based on the project creation information, and use the project identification field in the project file to determine the corresponding first visual modeling interface; or,

[0050] The project import information is obtained, the corresponding historical project file is determined based on the project import information, and the second visual modeling interface of the historical project is determined using the historical project identification field in the historical project file; the second visual modeling interface includes a historical visualization element.

[0051] In one possible implementation, the user can choose to create or import a project before entering the visual modeling interface;

[0052] When the user chooses to create a new project, project creation information is generated, a project file of the project is generated based on the project creation information, and a project identification field in the project file is used to determine the corresponding first visual modeling interface;

[0053] When the user chooses to import a project, project import information is generated, the corresponding historical project file is determined based on the project import information, and the second visual modeling interface of the historical project is determined using the historical project identification field in the historical project file.

[0054] The project identification field may refer to a project identifier (Identification, ID), and the corresponding project may be uniquely identified by the project ID.

[0055] For example, the corresponding sensor layout interface is determined according to the project ID in the project file corresponding to the sensor modeling project, and the corresponding decision algorithm diagram interface is determined according to the project ID corresponding to the algorithm modeling project; the historical project file may refer to the historical modeling file for the vehicle middleware.

[0056] In a possible implementation, the historical visualization elements may include, but are not limited to: historical component types, historical connection relationships, canvas layouts, etc., so that users can perform secondary editing and updating of historical project files.

[0057] Through the above method, the options of creating a new project and importing a historical project allow users to flexibly choose the operation mode, allowing users to create new projects or continue to process historical projects as needed, reducing operational limitations and improving the applicability of the system; reusing historical components helps to maintain the consistency of modeling data, and reusing pre-configured components and layouts can save a lot of time, achieve rapid iteration of vehicle middleware test solutions, and improve modeling efficiency.

[0058] In a specific embodiment, creating a visualization component type in a visualization modeling interface corresponding to a project is implemented based on the following steps:

[0059] The project file corresponding to the project is stored in the project table, and the project file includes the project name and project identification fields;

[0060] Determine the type of visualization component to be created based on the creation operation information, store the visualization component type in the component list, and store the project identification field in the project table as a foreign key in the component list; the foreign key is used to associate the visualization component type corresponding to the project;

[0061] A preset function is used to convert the newly added visualization component type in the component list into modeling data, so as to display the icon corresponding to the visualization component type in the visualization modeling interface.

[0062] In a possible implementation, the project table may refer to a project table stored in the system backend, in which the project files corresponding to each project may be stored; the component list may refer to the default property configuration information of the visual components of all projects.

[0063] Here, the project identification field in the project table is stored in the component list as a foreign key, so that each project in the project table can be associated with the default attribute configuration information of the corresponding visual component in the component list.

[0064] In a possible implementation, when a user chooses to import a project, based on the project ID corresponding to the project import information, the corresponding historical project file is determined in the project table, and a secondary check is performed on the historical project file to determine whether the historical project file conforms to a preset format;

[0065] If it matches, read the JSON file corresponding to the historical project file and write the data in the JSON file into the backend database;

[0066] If it does not match, a prompt message will be sent to the user to allow the user to reselect the import project.

[0067] The preset format check may refer to checking whether the historical project file includes data such as the project name, component type, and configuration data.

[0068] For example, Figure 2 As shown, Figure 2 The following is a flow chart of importing historical engineering files of a visual custom modeling method provided by an embodiment of the present invention, which includes the following steps:

[0069] Step S201, selecting the historical project file to be imported;

[0070] Step S202, file decompression;

[0071] Step S203, secondary check (i.e. preset format check), if passed, proceed to step S204, if not, proceed to step S205;

[0072] Here, check whether the historical project file contains data such as project name, component type, configuration data, etc.

[0073] Step S204, reading historical project files;

[0074] Here, read the JSON file corresponding to the historical project file

[0075] Step S205, sending prompt information;

[0076] Step S206, writing into the database;

[0077] Here, the data in the JSON file is written to the backend database.

[0078] In a possible implementation, a preset function is used to convert the newly added visualization component type in the component list into modeling data, so as to display the icon corresponding to the visualization component type in the visualization modeling interface, including:

[0079] The parseData function is used to parse and convert the newly added visualization component types and related data into a format that the system can understand and process, that is, modeling data;

[0080] The modeling data is rendered on the canvas, the modeling data is converted into visualization elements and drawn on the canvas, so as to display icons corresponding to the visualization element types in the visualization modeling interface.

[0081] The parseData function can be used to parse and process the input data, converting the original data into a structure or format that is easier to use; the modeling data may refer to the data parsed by the parseData function, and may be the core data structure used for rendering and displaying in the visual modeling interface.

[0082] For example, components such as lidar and cameras are displayed in graphical form in the sensor modeling interface.

[0083] Furthermore, the canvas rendering of the modeling data may be an independent rendering.

[0084] Here, independent rendering may mean that the parsed modeling data can be displayed independently of other data, that is, the display of the visualization component type does not depend on other data states and can be rendered as an independent module.

[0085] Furthermore, since the visualization component type is rendered independently, users can easily operate it without affecting the display of other parts; therefore, the visualization component type can allow users to perform further interactions, such as dragging and adjusting properties.

[0086] In a possible implementation, the icon forms of visualization components can be combined and personalized, and users can upload graphic shapes simultaneously when adding new visualization component types, thereby increasing the diversity of visualization component icon displays.

[0087] Through the above method, the project file is stored in the project table, so that the project name and project identification fields are structured, thereby providing centralized management of the project and facilitating subsequent queries and operations; by creating visual component types and storing them in the component list, the system can flexibly support different types of components, thereby enhancing the user's selectivity and flexibility in the modeling process; users can operate through visual elements such as component icons of the vehicle middleware, which lowers the user's understanding threshold of middleware modeling and makes the modeling process more friendly and easy to use.

[0088] In a specific embodiment, obtaining configuration operation information for a visualization element type, configuring properties of the visualization element type based on the configuration operation information, and rendering the property configuration result of the visualization element in real time in a visualization modeling interface include:

[0089] Based on the identification field of the visualization component type in the configuration operation information, a preset query interface is used to obtain basic configuration information of the visualization component type; the basic configuration information is stored in a preset data structure of the global state management;

[0090] Send the basic configuration information to the dynamic form corresponding to the visual modeling interface, and use the dynamic form to determine the corresponding rendering component based on the form item type field in the basic configuration information;

[0091] The rendering component is used to modify the field data of the basic configuration information, determine the updated configuration information, render the attribute configuration results corresponding to the updated configuration information in real time in the visual modeling interface, and store the updated configuration information in a preset data structure.

[0092] In one possible implementation, the configuration operation information may refer to a combination of information corresponding to a click event generated when a user clicks on an icon representing a type of visualization element in a visualization modeling interface; the configuration operation information may include, but is not limited to: an identification field of the clicked element, a timestamp corresponding to the click event, and user context information.

[0093] Among them, the click event can be used to represent the user's configuration intention for a specific visualization element.

[0094] Further, based on the identification field of the visualization element type in the configuration operation information, the basic configuration information of the visualization element type is obtained by using a preset query interface, which may include:

[0095] The identification field of the currently clicked visualization component in the configuration operation information is obtained, and the preset query interface findById is used for querying. The basic configuration information of the clicked visualization component is obtained by searching through the representation field in the component list.

[0096] Here, the preset data structure in which the basic configuration information is stored in the global state management may be stored in settingData; settingData may refer to a data structure format for caching data.

[0097] Exemplarily, the basic configuration information may include: basic configuration, appearance configuration and registration events.

[0098] In a possible implementation, a rendering component is used to modify the field data of the basic configuration information, determine the updated configuration information, render the attribute configuration result corresponding to the updated configuration information in real time in the visual modeling interface, and store the updated configuration information in a preset data structure, based on the following steps:

[0099] Use dynamic forms to obtain basic configuration information, determine the corresponding form configuration through key matching of the basic configuration information, determine the rendered component through elementType in the form configuration, and render the value corresponding to the key into the visual component.

[0100] Here, dynamic form may refer to a form component that dynamically generates and renders form items based on runtime data or configuration; form configuration is setting information used to define the structure and behavior of each form item in the dynamic form; each attribute in the basic configuration information has a corresponding key-value pair, where the key is used to identify the specific attribute name, and the value is used to identify the actual content of the attribute.

[0101] Furthermore, the elementType in the form configuration is used to define the type of the form item. For example, the elementType may include but is not limited to: a text box (input), a drop-down menu (select), a check box (checkbox), and the like.

[0102] For example, add or modify the name of a sensor component through the text box, and modify the type of the added sensor component through the check box.

[0103] In a possible implementation, a rendering component is used to modify the field data of the basic configuration information, determine the updated configuration information, render the attribute configuration result corresponding to the updated configuration information in real time in the visual modeling interface, and store the updated configuration information in a preset data structure, based on the following steps:

[0104] After determining the rendering component corresponding to the visualization component, the rendering component is used to modify the field data of the basic configuration information, and the updated configuration information (that is, the modified basic configuration information) is stored in settingData;

[0105] Call the update API to save the updated configuration information to the backend storage.

[0106] Here, storing the updated configuration information in settingData may mean temporarily storing the data in the system front end for real-time rendering of the property configuration structure of the visualization components of the visual modeling interface; calling the update interface to save the updated configuration information to the backend storage may be used to store the updated configuration information.

[0107] Through the above method, through the preset query interface and the global state management system, the user no longer needs to manually enter the information of each configuration item. The basic configuration information can be automatically obtained from the preset data structure and sent to the dynamic form corresponding to the interface, which greatly reduces the complexity and time consumption of manual configuration; after the user modifies the configuration information in the visual modeling interface, the system can render the updated results in real time, so that the user can immediately see the effect of the changes made, which increases the ease of use of middleware modeling and improves the user's real-time interactive experience.

[0108] In a specific embodiment, the method further comprises:

[0109] Acquire update configuration information in a preset data structure, and convert the update configuration information into a preset format using a mapping dictionary to convert field data of the update configuration information into a modeling tool format;

[0110] The updated configuration information converted into the modeling tool format is stored in a preset data structure, and an update interface is used to update the visual components in the visual modeling interface based on the updated configuration information in the preset data structure.

[0111] Specifically, obtain the updated configuration information from settingData, call the data conversion method, inject the configuration items and property settings corresponding to the updated configuration information into the conversion method, and select the corresponding mapping dictionary according to the type currently being modified;

[0112] Use mapping dictionaries to convert fields in the updated configuration information into the specific data format required by the modeling tool;

[0113] Save the updated configuration information after the conversion format back to settingData for subsequent operations.

[0114] For example, suppose settingData stores the following update configuration information: settingData = {

[0115] color:'#ff0000',

[0116] size:'large',

[0117] shape:'circle',

[0118] }

[0119] Use the mapping dictionary to convert the fields in the updated configuration information into the following specific data format:

[0120] {

[0121] "fill":"#ff0000",

[0122] "width":100,

[0123] "height":100,

[0124] "type":"circle"

[0125] }

[0126] Through the above method, the format of the updated configuration information is converted through the mapping dictionary, which can ensure that the data input by the user meets the data structure and format requirements required by the modeling tool, and improve the compatibility between different data formats; due to the use of the mapping dictionary, the configuration update and data conversion process is more universal, avoiding the cumbersome hard coding process, thereby improving the efficiency of vehicle middleware modeling and reducing misunderstandings and errors in the middleware development process.

[0127] In a specific embodiment, the drag operation information for the visualization element type is obtained, and the corresponding modeling data is determined based on the drag operation information in the visualization modeling interface, which is implemented based on the following steps:

[0128] When first drag operation information for a component type is detected, first configuration information corresponding to the component type is acquired, and the configuration information of the component is stored in a first association table;

[0129] When second drag operation information for the pile type is detected, second configuration information corresponding to the pile type is obtained, and the second configuration information is stored in a second association table; the second association table is associated with the first association table;

[0130] When third drag operation information for the connection type is detected, third configuration information corresponding to the connection type is acquired, and the third configuration information is stored in a third association table; the third association table is associated with the second association table;

[0131] Based on the drag operation results corresponding to the first drag operation information, the second drag operation information, and the third drag operation information, corresponding modeling data is determined.

[0132] Specifically, when the user obtains the first drag information of the component type in the visual modeling interface, the component type is instantiated and the first configuration information corresponding to the component type is stored in the first association table; when the user obtains the second drag information of the pile type in the visual modeling interface, the pile type is instantiated and the second configuration information corresponding to the pile type is stored in the second association table; when the user obtains the third drag information of the connection type in the visual modeling interface, the connection type is instantiated and the third configuration information corresponding to the connection type is stored in the third association table.

[0133] Here, obtaining the first drag information may refer to detecting that the user drags the component type into the canvas of the visual modeling interface; similarly, obtaining the second drag information may refer to detecting that the user drags the pile type into the canvas of the visual modeling interface, and obtaining the third drag information may refer to detecting that the user drags the connection type into the canvas of the visual modeling interface.

[0134] In a possible implementation, data in the first association table is inherited from the component table in the component list, data in the second association table is inherited from the pile table in the component list, and data in the third association table is inherited from the connection table in the component list.

[0135] Here, the component table may refer to the part of the component list storing the identification field of the component, the pile table may refer to the part of the component list storing the identification field of the pile, and the node table may refer to the part of the component list storing the identification field of the node.

[0136] For example, Figure 3 is an associated representation diagram of a visual custom modeling method provided by an embodiment of the present invention, wherein:

[0137] The data in the first association table is inherited from the component table and the canvas table, the data in the second association table is inherited from the pile table and the first association table, and the data in the third association table is inherited from the connection table and the second association table.

[0138] Further, assuming that there are N data stored in the component table, pile table, connection table and canvas table respectively, any one of the N components in the component table and the N canvases in the canvas table can generate corresponding data in the N first association tables, and so on. The relationship between the tables is a 1:N relationship.

[0139] Through the above method, by capturing the information of three drag operations, the configuration information of each component type is stored in different associated tables, and the association between these tables is ensured, so that the relationship between components, piles and connections is clear and traceable, which is convenient for developers to effectively understand the interaction relationship between various visual components in the vehicle middleware, and can effectively manage complex modeling data structures, thereby improving the accuracy of autonomous driving development; whenever the user performs a drag operation in the interface, the system automatically generates and stores the corresponding modeling data, and associates them, realizing real-time and dynamic generation of a complete model, thereby supporting a variety of autonomous driving scenarios and application requirements, and enhancing the adaptability of the system.

[0140] In a specific embodiment, obtaining first configuration information corresponding to a component type and storing the configuration information of the component in a first association table includes:

[0141] Using a preset query interface, obtaining first configuration information corresponding to the component type based on the first identification field of the component type, and storing the first identification field and the first configuration information in a first association table;

[0142] Obtaining second configuration information corresponding to the pile type and storing the second configuration information in a second association table includes:

[0143] Using a preset query interface, obtaining second configuration information corresponding to the pile type based on the second identification field of the pile type, and storing the second identification field and the second configuration information in a second association table;

[0144] Acquiring third configuration information corresponding to the connection type, and storing the third configuration information in a third association table, including:

[0145] A preset query interface is used to obtain third configuration information corresponding to the connection type based on the third identification field of the connection type, and the third identification field and the third configuration information are stored in a third association table.

[0146] Specifically, based on the first drag information, the findbyID interface is used to query the first configuration information of the corresponding component in the component list, and the first identification field and the first configuration information are stored in the first association table, with the first identification field serving as the foreign key of the first association table; based on the second drag information, the findbyID interface is used to query the second configuration information of the corresponding pile in the component list, and the second identification field and the second configuration information are stored in the second association table, with the second identification field serving as the foreign key of the second association table; based on the third drag information, the findbyID interface is used to query the third configuration information of the corresponding connection in the component list, and the third identification field and the third configuration information are stored in the third association table, with the third identification field serving as the foreign key of the third association table.

[0147] Further, when it is detected that the user modifies the property configuration of the visualization element, the first configuration information, the second configuration information or the third configuration information stored in the first association table, the second association table and the third association table are modified accordingly.

[0148] Furthermore, the visualization components, canvas types and canvas rules in the project are packaged and processed, and the data in the project is converted and exported in a preset format using mapping dictionaries corresponding to different types. All data are merged and output as a JSON file for use by the debugging engine or other modeling tools.

[0149] For example, Figure 4 The following is a flow chart of an export process of a visual custom modeling method provided by an embodiment of the present invention, comprising the following steps:

[0150] Step S401, connecting to a database;

[0151] Here, connect to the component list to obtain the configuration information of the visualization components of the current project;

[0152] Step S401, obtaining current project data;

[0153] Here, the current engineering data is determined based on the configuration information of the visualization component;

[0154] Step S401, generating a JSON file;

[0155] Here, the mapping dictionary is used to convert the current project data into a preset format and generate a JSON file in a preset format;

[0156] Step S401, compressing the JSON file;

[0157] Step S401, select a file storage path.

[0158] In a specific embodiment, Figure 5 is an overall architecture diagram of a visual custom modeling method provided by an embodiment of the present invention, such as Figure 5 As shown, where:

[0159] The visual modeling interface of the project is opened according to the user's creation or import. When the user chooses to import, the engineering data of the historical project is imported; the user creates a visual component type in the visual modeling interface corresponding to the project, such as Figure 5The piles, components, connections, canvases and canvas rules in the visualization modeling interface; users configure the properties of the created visualization component type, and convert the configuration results into a preset format in data processing (i.e., using a mapping dictionary). The configuration results can be rendered in real time in the visualization modeling interface through a graphical interface (i.e., a dynamic form), and the configuration information is stored in the configuration data (i.e., a component list). The configuration data can be saved and exported, and the exported configuration data can be used as a project file; according to the visualization component corresponding to the drag information, the updated image is displayed on the visualization interface through the image interface; the mapping dictionary is used to convert the visualization component into a preset format and generate a project file (i.e., export the project); through the scan data format and the scan data file, users can be allowed to import external files and data, parse the file based on the preset scan format, and convert it into project data; by creating simulation data, the function of the modeling system can be tested.

[0160] Figure 6 is a structural diagram of a visual custom modeling device provided by an embodiment of the present invention, such as Figure 6 As shown, the device can be applied to intelligent electronic devices such as servers and computers; the device includes: a component creation module 601, a property configuration module 602 and a modeling export module 603;

[0161] The component creation module 601 is used to obtain creation operation information and create a visual component type in the visual modeling interface corresponding to the project; the visual component type includes: canvas rule, canvas type, component type, pile type, and connection type;

[0162] The property configuration module 602 is used to obtain configuration operation information of the visualization element type, perform property configuration on the visualization element type based on the configuration operation information, and render the property configuration result of the visualization element in real time in the visualization modeling interface;

[0163] The modeling export module 603 is used to obtain the drag operation information for the visualization element type, determine the corresponding modeling data based on the drag operation information in the visualization modeling interface, and use the mapping dictionary to convert and export the modeling data into a preset format.

[0164] In a specific embodiment, the apparatus further includes an engineering determination module 604;

[0165] The project determination module 604 is used to obtain project creation information, generate a project file of the project based on the project creation information, and use the project identification field in the project file to determine the corresponding first visual modeling interface; or

[0166] The project import information is obtained, the corresponding historical project file is determined based on the project import information, and the second visual modeling interface of the historical project is determined using the historical project identification field in the historical project file; the second visual modeling interface includes a historical visualization element.

[0167] In a specific embodiment, the component creation module 601 is used to store a project file corresponding to a project into a project table, where the project file includes a project name and a project identification field;

[0168] Determine the type of visualization component to be created based on the creation operation information, store the visualization component type in the component list, and store the project identification field in the project table as a foreign key in the component list; the foreign key is used to associate the visualization component type corresponding to the project;

[0169] A preset function is used to convert the newly added visualization component type in the component list into modeling data, so as to display the icon corresponding to the visualization component type in the visualization modeling interface.

[0170] In a specific embodiment, the attribute configuration module 602 is used to obtain basic configuration information of the visualization element type using a preset query interface based on the identification field of the visualization element type in the configuration operation information; the basic configuration information is stored in a preset data structure of the global state management;

[0171] Send the basic configuration information to the dynamic form corresponding to the visual modeling interface, and use the dynamic form to determine the corresponding rendering component based on the form item type field in the basic configuration information;

[0172] The rendering component is used to modify the field data of the basic configuration information, determine the updated configuration information, render the attribute configuration results corresponding to the updated configuration information in real time in the visual modeling interface, and store the updated configuration information in a preset data structure.

[0173] In one specific embodiment, the format conversion module 605;

[0174] The format conversion module 605 is used to obtain the update configuration information in the preset data structure, and convert the update configuration information into a preset format using a mapping dictionary to convert the field data of the update configuration information into a modeling tool format;

[0175] The updated configuration information converted into the modeling tool format is stored in a preset data structure, and an update interface is used to update the visual components in the visual modeling interface based on the updated configuration information in the preset data structure.

[0176] In a specific embodiment, the modeling export module 603 is used to obtain first configuration information corresponding to the component type when first drag operation information for the component type is detected, and store the configuration information of the component in a first association table;

[0177] When second drag operation information for the pile type is detected, second configuration information corresponding to the pile type is obtained, and the second configuration information is stored in a second association table; the second association table is associated with the first association table;

[0178] When third drag operation information for the connection type is detected, third configuration information corresponding to the connection type is acquired, and the third configuration information is stored in a third association table; the third association table is associated with the second association table;

[0179] Based on the drag operation results corresponding to the first drag operation information, the second drag operation information, and the third drag operation information, corresponding modeling data is determined.

[0180] In a specific embodiment, the modeling export module 603 is used to use a preset query interface to obtain first configuration information corresponding to the component type based on the first identification field of the component type, and store the first identification field and the first configuration information in a first association table;

[0181] Using a preset query interface, obtaining second configuration information corresponding to the pile type based on the second identification field of the pile type, and storing the second identification field and the second configuration information in a second association table;

[0182] A preset query interface is used to obtain third configuration information corresponding to the connection type based on the third identification field of the connection type, and the third identification field and the third configuration information are stored in a third association table.

[0183] It should be noted that: the visual custom modeling device provided in the above embodiment only uses the division of the above program modules as an example to illustrate when implementing the corresponding visual custom modeling method. In actual applications, the above processing can be assigned to different program modules as needed, that is, the internal structure of the device can be divided into different program modules to complete all or part of the above-described processing. Figure 1 The embodiments of the method shown belong to the same concept, and the specific implementation process is detailed in the method embodiments, which will not be repeated here.

[0184] The embodiment of the present invention also provides a computer device having the above Figure 6 Visualization of the custom modeling device shown.

[0185] See also Figure 7 , Figure 7 is a schematic diagram of the structure of another visual custom modeling device provided by an embodiment of the present invention, such as Figure 7As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 7 A processor 10 is taken as an example.

[0186] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0187] The memory 20 stores instructions executable by at least one processor 10, so that the at least one processor 10 executes the method shown in the above embodiment.

[0188] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0189] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0190] The computer device further comprises a communication interface 30 for the computer device to communicate with other devices or a communication network.

[0191] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0192] A part of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the existence of the computer program instruction in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc., and accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium accessible to the computer.

[0193] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A visual custom modeling method, characterized in that: The method comprises: Acquire creation operation information, and create a visualization component type in a visualization modeling interface corresponding to the project; the visualization component type includes: canvas rule, canvas type, component type, pile type, and connection type; Acquire configuration operation information of the visualization element type, perform attribute configuration on the visualization element type based on the configuration operation information, and render the attribute configuration result of the visualization element in real time in the visualization modeling interface; Drag operation information for the visualization element type is obtained, corresponding modeling data is determined in the visualization modeling interface based on the drag operation information, and a mapping dictionary is used to convert and export the modeling data into a preset format.

2. The method according to claim 1, characterized in that The method further comprises: Acquire project creation information, generate a project file of the project based on the project creation information, and use a project identification field in the project file to determine a corresponding first visual modeling interface; or, The project import information is obtained, the corresponding historical project file is determined based on the project import information, and the second visual modeling interface of the historical project is determined using the historical project identification field in the historical project file; the second visual modeling interface includes a historical visualization element.

3. The method according to claim 2, characterized in that Create a visualization component type in the visualization modeling interface corresponding to the project based on the following steps: The project file corresponding to the project is stored in the project table, wherein the project file includes a project name and a project identification field; Determine the type of visualization component to be created based on the creation operation information, store the visualization component type in a component list, and store the project identification field in the project table as a foreign key in the component list; The foreign key is used to associate the visualization component type corresponding to the project; The preset function is used to convert the newly added visualization component type in the component list into modeling data, so as to display the icon corresponding to the visualization component type in the visualization modeling interface.

4. The method according to claim 3, characterized in that The obtaining of configuration operation information of the visualization element type, performing attribute configuration on the visualization element type based on the configuration operation information, and rendering the attribute configuration result of the visualization element in real time in the visualization modeling interface includes: Based on the identification field of the visualization element type in the configuration operation information, a preset query interface is used to obtain basic configuration information of the visualization element type; the basic configuration information is stored in a preset data structure of the global state management; Sending the basic configuration information to a dynamic form corresponding to the visual modeling interface, and using the dynamic form to determine a corresponding rendering component based on a form item type field in the basic configuration information; The rendering component is used to modify the field data of the basic configuration information, determine the updated configuration information, render the attribute configuration result corresponding to the updated configuration information in real time in the visual modeling interface, and store the updated configuration information in the preset data structure.

5. The method according to claim 4, characterized in that The method further comprises: Acquire update configuration information in a preset data structure, and convert the update configuration information into a preset format using a mapping dictionary to convert field data of the update configuration information into a modeling tool format; The updated configuration information converted into the modeling tool format is stored in the preset data structure, and the visual element in the visual modeling interface is updated based on the updated configuration information in the preset data structure using an update interface.

6. The method according to claim 5, characterized in that Obtaining drag operation information for the visualization element type, and determining corresponding modeling data based on the drag operation information in the visualization modeling interface, is achieved based on the following steps: When first drag operation information for a component type is detected, first configuration information corresponding to the component type is acquired, and the configuration information of the component is stored in a first association table; When second drag operation information for a pile type is detected, second configuration information corresponding to the pile type is acquired, and the second configuration information is stored in a second association table; The second association table is associated with the first association table; When third drag operation information for a connection type is detected, third configuration information corresponding to the connection type is acquired, and the third configuration information is stored in a third association table; The third association table is associated with the second association table; Based on the drag operation results corresponding to the first drag operation information, the second drag operation information, and the third drag operation information, corresponding modeling data is determined.

7. The method according to claim 6, characterized in that The obtaining first configuration information corresponding to the component type and storing the configuration information of the component in a first association table includes: Using a preset query interface, based on a first identification field of the component type, obtaining first configuration information corresponding to the component type, and storing the first identification field and the first configuration information in a first association table; The obtaining the second configuration information corresponding to the pile type and storing the second configuration information in a second association table includes: Using the preset query interface, based on the second identification field of the pile type, obtaining the second configuration information corresponding to the pile type, and storing the second identification field and the second configuration information in a second association table; The acquiring the third configuration information corresponding to the connection type and storing the third configuration information in a third association table includes: The preset query interface is adopted to obtain third configuration information corresponding to the connection type based on the third identification field of the connection type, and the third identification field and the third configuration information are stored in a third association table.

8. A visual custom modeling device, characterized in that: The device comprises: The component creation module is used to obtain creation operation information and create a visual component type in the visual modeling interface corresponding to the project; the visual component type includes: canvas rule, canvas type, component type, pile type, and connection type; An attribute configuration module, used to obtain configuration operation information of the visualization element type, perform attribute configuration on the visualization element type based on the configuration operation information, and render the attribute configuration result of the visualization element in real time in the visualization modeling interface; The modeling export module is used to obtain the drag operation information for the visualization element type, determine the corresponding modeling data based on the drag operation information in the visualization modeling interface, and use a mapping dictionary to convert and export the modeling data into a preset format.

9. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the visual custom modeling method according to any one of claims 1 to 7 by executing the computer instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the visual custom modeling method according to any one of claims 1 to 7.

Images