Interactive immersive design system and method for production environment and equipment layout

Abstract

The present invention belongs to the technical field of production line design. The present invention provides a production environment and equipment layout interactive immersive design system and method. First, the modeling, standardization and model management of the production line equipment are completed according to the model processing end; then, in response to the design instructions of the designer in the three-dimensional design end, the model is retrieved from the model processing end through the data management end, and a three-dimensional production environment scene is created in the three-dimensional design end; then, the operation information of each operation client is reviewed and optimized for the three-dimensional production environment scene; finally, according to the operation information of each operation client, the status information of the three-dimensional production environment scene in the data management end is updated and broadcast to each operation client. The present invention adds a VR review end for multi-terminal collaborative design, which not only solves the shortcomings of conventional two-dimensional design, but also takes into account the requirements of collaborative design. At the same time, the immersive editing and review method reduces the professional requirements for the owner, effectively improving the design interaction efficiency and the quality of design results.

Description

Technical Field

[0001] The present invention belongs to the technical field of production line design, and in particular relates to a system and method for interactive immersive design of production environment and equipment layout. Background Art

[0002] In the manufacturing industry and other fields, production line layout design is a crucial link. With the increasing requirements for refinement and efficiency of industrial production, the traditional way of production line layout based on experience alone can no longer meet the needs. The production line layout design technology that combines on-site layout with drawings has emerged.

[0003] In the process of construction, transformation and upgrading of production lines and production environments in traditional manufacturing machinery processing enterprises, the conventional means are still to first use a combination of on-site layout and drawings to design the production line layout. Different professional designers draw two-dimensional drawings of the corresponding system. After the design is completed, the drawings are associated and integrated, and a three-dimensional model is established based on the two-dimensional drawings. The model is used to perform a three-dimensional space rationality check and analysis. After problems are found and recorded, the drawings are modified and changed. At the same time, a production line operation animation is made to communicate and confirm with the user. This repeated iteration finally completes the production line design. Not only is the process cumbersome, but the models and drawings are also prone to errors and have a low accuracy rate.

[0004] The current technical methods have the following problems:

[0005] First, the plane positioning accuracy of on-site layout and two-dimensional drawing design is relatively low, and the spatial layout is prone to cross-interference and influence;

[0006] Second, it is easy to make mistakes in the three-dimensional space control, especially in large workshops, complex equipment, large equipment that is difficult to move, a large number of supporting pipelines and logistics transmission equipment. The traditional layout design method will have problems such as "errors, omissions, collisions, and shortages";

[0007] Third, the current mode of verification through 3D modeling for 3D space control is not linked with the design process, and the problem association and processing efficiency are low;

[0008] Fourth, the degree of data sharing among designers of different disciplines is low, the design collaboration and interaction is poor, and the design results are not intuitive;

[0009] Fifth, industrial production line design drawings are highly professional, and users face knowledge barriers. At the same time, they interact through produced process animations, and the entire design process does not involve enough people, which seriously affects design efficiency and quality and wastes manpower and material resources.

[0010] Virtual Reality (VR) is a computer science technology that brings users' vision and hearing into a virtual environment through equipment and software, and enables users to immerse themselves in and interact with virtual scenes with the help of touch and smell, ensuring the user's immersion, interactivity and imagination. The sense of immersion is achieved through technical means such as high-resolution images and stereo sound effects, providing users with a "sense of reality". At present, VR technology has been widely used in military aerospace, game entertainment, and urban planning. The present invention uses VR technology to improve the traditional production environment and equipment layout. Summary of the invention

[0011] In view of this, the present invention provides a system and method for interactive immersive design of production environment and equipment layout, aiming to solve the problems of complicated production line design and low accuracy in the prior art.

[0012] A first aspect of an embodiment of the present invention provides a production environment and equipment layout interactive immersive design system, including:

[0013] VR review end, 3D design end, model processing end and data management end; VR review end, 3D design end and model processing end are connected to the data management end respectively;

[0014] The model processing end is used to complete the modeling, standardization and model management of production line equipment;

[0015] The 3D design end is used to respond to the design instructions of the designer, retrieve the model from the model processing end through the data management end, and create a 3D production environment scene;

[0016] The VR review end includes multiple operation clients; each operation client reviews and optimizes the three-dimensional production environment scene in response to the operation information;

[0017] The data management end is used to update the status information of the three-dimensional production environment scene according to the operation information of each operation client, and broadcast the status information to each operation client.

[0018] In a possible implementation, the operating client is used to:

[0019] Roam, query data, and mark information in the 3D production environment to complete the review process;

[0020] Perform interactive design operations on the 3D production environment scene to complete the optimization process.

[0021] In a possible implementation, the operation client is further used to:

[0022] Display the status information corresponding to each operation client in the 3D production environment scene with different tags;

[0023] According to the status information corresponding to each operation client, the adjustment information of the three-dimensional production environment scene is determined.

[0024] In a possible implementation, the data management end is further used for:

[0025] According to the adjustment information, the three-dimensional production environment scene is updated, and the 2D drawings corresponding to the three-dimensional production environment scene are updated at the same time.

[0026] In a possible implementation, the 3D design end is used to:

[0027] In response to the design instructions of the designer, the model is retrieved from the model processing end through the data management end to complete any one or more of the following spatial operations: model import, material editing, attribute binding to rotate, translate;

[0028] The completed model is subjected to collision analysis, interface adsorption, and measurement and annotation to obtain a three-dimensional production environment scene.

[0029] In a possible implementation, the three-dimensional design end includes a plurality of design terminals; the three-dimensional design end is used for:

[0030] According to the design instructions of each design terminal, the model is retrieved from the model processing end through the data management end to create a three-dimensional production environment scene.

[0031] In one possible implementation, the model processing end is used to:

[0032] Build models for each production line equipment;

[0033] Import models of third-party devices;

[0034] All models are lightweight and annotated.

[0035] In one possible implementation, the model processing end is used to:

[0036] Classify and call each model.

[0037] In one possible implementation, the production line equipment includes machining equipment and transmission equipment.

[0038] A second aspect of an embodiment of the present invention provides a production environment and equipment layout interactive immersive design method, which is applied to the production environment and equipment layout interactive immersive design system of the first aspect above; the method comprises:

[0039] Complete the modeling, standardization and model management of production line equipment based on the model processing end;

[0040] In response to the design instructions of the designer in the 3D design end, the model is retrieved from the model processing end through the data management end, and a 3D production environment scene is created in the 3D design end;

[0041] Review and optimize the three-dimensional production environment scene in response to the operation information of the VR review terminal including each operation client;

[0042] According to the operation information of each operation client, the status information of the three-dimensional production environment scene in the data management terminal is updated, and the status information is broadcast to each operation client.

[0043] The interactive immersive design system and method for production environment and equipment layout provided by the embodiment of the present invention first completes the modeling, standardization and model management of the production line equipment according to the model processing end; then responds to the design instructions of the designer in the three-dimensional design end, retrieves the model from the model processing end through the data management end, and creates a three-dimensional production environment scene in the three-dimensional design end; then responds to the VR review end including the operation information of each operation client to review and optimize the three-dimensional production environment scene; finally, according to the operation information of each operation client, updates the status information of the three-dimensional production environment scene in the data management end, and broadcasts the status information to each operation client. The present invention solves the shortcomings of conventional two-dimensional design and takes into account the requirements of collaborative design by adding a VR review end for multi-terminal collaborative design. At the same time, the immersive editing and review method reduces the professional requirements for the owner, effectively improving the efficiency of design interaction and the quality of design results. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0045] Figure 1 It is a structural schematic diagram of a production environment and equipment layout interactive immersive design system provided by an embodiment of the present invention;

[0046] Figure 2 It is a flow chart for implementing the interactive immersive design method for production environment and equipment layout provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0047] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present invention. However, it should be clear to those skilled in the art that the present invention may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present invention.

[0048] Figure 1 Schematic diagram of the structure of the interactive immersive design system for production environment and equipment layout provided by an embodiment of the present invention. Figure 1 As shown, the interactive immersive design system for production environment and equipment layout includes: a VR review terminal 11, a three-dimensional design terminal 12, a model processing terminal 13 and a data management terminal 14; the VR review terminal 11, the three-dimensional design terminal 12, and the model processing terminal 13 are respectively connected to the data management terminal 14;

[0049] The model processing end is used to complete the modeling, standardization and model management of production line equipment; the 3D design end is used to respond to the design instructions of the designer, retrieve the model from the model processing end through the data management end, and create a 3D production environment scene; the VR review end includes multiple operation clients; each operation client reviews and optimizes the 3D production environment scene in response to the operation information; the data management end is used to update the status information of the 3D production environment scene according to the operation information of each operation client, and broadcast the status information to each operation client.

[0050] The VR review end consists of two parts: hardware and software. The hardware part includes VR head display equipment, VR operating handle and VR basic platform. The software part includes data hot synchronization update module, interactive editing control module, scene roaming review management module, viewpoint management module, scene data processing module and first auxiliary design module. The three-dimensional design end includes scene solution management module, model editing management module, three-dimensional layout design module, solution results export module and second auxiliary design module. The model processing end includes model lightweight processing module, model standardization processing module, model interface definition module and third-party model import module. The data management end includes three-dimensional digital model management module and update collaborative management module.

[0051] In some embodiments, the model processing end is used to: establish models of various production line equipment; import models of third-party equipment; and perform lightweight processing and annotation on all models.

[0052] In the embodiment of the present invention, the modeling of the production line equipment is the starting step of the entire design system. In this process, the production line equipment model is constructed with the help of professional modeling software such as Revit and 3DMAX. These software have powerful modeling functions and can accurately create three-dimensional models according to the detailed design parameters, appearance characteristics and functional requirements of the production line equipment. Designers use various tools of these software, such as the family creation function in Revit and the polygon modeling tool in 3DMAX, to carefully shape every part of the production line equipment, from mechanical structure to appearance details, to ensure that the model can accurately reflect the form and characteristics of the actual equipment, and provide a high-quality basic model for subsequent design work.

[0053] In some embodiments, the model processing end is used to classify and call various models.

[0054] In an embodiment of the present invention, for third-party models, whether they are imported offline or imported in batches through an API interface, lightweight processing must be performed. In this process, a variety of advanced methods and technologies are used, such as model geometric complexity simplification technology, to analyze and optimize the geometric shape of the model, remove unnecessary details and complex structures, and reduce the geometric complexity of the model while ensuring the core features of the model. At the same time, the model texture and material optimization method is used to adjust the texture mapping and material properties of the model, and reduce the amount of texture data and material calculation without affecting the visual effect. In addition, data compression technology is also used to compress the overall data of the model to further reduce the size of the model file. Through these lightweight processing methods, the performance burden of the design computer is effectively reduced, and the impact of network bandwidth on model loading and display is reduced, ensuring that the model can be smoothly applied in subsequent design and review links, and even in a relatively low-performance computer or network environment, a good visual experience and operating performance can be obtained.

[0055] In addition to lightweight processing, model standardization is equally important. Standardize the model format, scale, etc. and convert them. Models from different sources and formats are uniformly converted into the standard format specified by the system to ensure the compatibility of the model between various modules. At the same time, a series of interfaces and rules are defined based on the business attributes of the model. For example, for model mutual exclusion, it is clearly stipulated which models cannot exist or operate at the same time in a specific scenario; for model links, the connection methods and rules between models are defined; considering the safety distance factor, reasonable safety distance requirements are set for the layout of the model in the production environment. These interfaces and rules provide clear guidance and constraints for subsequent design and review work.

[0056] The model warehousing management link conducts detailed classification management of various models. This includes production line environment models, machining center, sawing machine, broaching machine, grinder, planer, boring machine, milling machine, drilling machine and other machining equipment models, as well as transmission equipment models such as conveyor belts. Models are classified and stored according to factors such as equipment type, function, and position in the production process. For example, all machining center models are classified into one category, and different types of drilling machine models are placed under the corresponding drilling machine category. This classification method helps to find the required models quickly and accurately, and improves model management efficiency. Through a carefully designed data interface, the model library can provide convenient model calling services for production line 3D design and immersive interactive design modules. When designers need to use a specific model during 3D design or review, they only need to send a request to the model library through the corresponding data interface, and the model library can quickly and accurately provide the required model, ensuring the smooth progress of the design work and realizing the efficient use and sharing of model resources.

[0057] In some embodiments, the operating client is used to: roam, query data, and annotate information on the three-dimensional production environment scene to complete the review process; and perform interactive design operations on the three-dimensional production environment scene to complete the optimization process.

[0058] In some embodiments, the three-dimensional design end is used to: respond to the design instructions of the designer, call the model from the model processing end through the data management end, and complete any one or more of the following spatial operations: model import, material editing, attribute binding for rotation, translation; perform collision analysis, interface adsorption, measurement and annotation on the model that has completed the operation to obtain a three-dimensional production environment scene.

[0059] In an embodiment of the present invention, in the three-dimensional design step of the production line, creating a three-dimensional production environment scene is the core link. This process requires the participation of multi-professional designers, who work together in the same virtual work scene to create a reasonable and efficient production environment. This unified work scene provides a platform for communication and collaboration for designers from different professional backgrounds. Mechanical design professionals can start from the perspective of the physical structure and mechanical properties of the equipment, process design professionals can plan according to the production process and process requirements, and professionals such as electrical design and automation control design can also play their respective professional advantages, integrating various design elements together to form a three-dimensional space that comprehensively and accurately reflects the actual production situation.

[0060] Designers first need to obtain previously processed and managed models from the model library. Through the data interface designed by the system, they can easily import the required models into the 3D design scene. These models cover production line environment models, various machining equipment models (such as machining centers, saws, broaching machines, grinders, planers, boring machines, milling machines, drilling machines, etc.) and transmission equipment models such as conveyors. The imported models are the basic elements of the entire design. They carry key information such as the geometry, size, and structure of the equipment, and provide visual objects for subsequent design operations.

[0061] After importing the model, material editing is an important step to make the model closer to the actual production environment. Designers need to adjust the material of the model according to the real material properties of the equipment. For example, for equipment made of metal materials, it is necessary to set appropriate parameters such as glossiness, reflectivity, and color to make it visually present the texture of metal; for rubber or plastic parts such as conveyor belts, it is necessary to adjust its softness, texture and other material properties. Through accurate material editing, not only can the realism of the model be improved, but it can also allow designers and other relevant personnel to more clearly understand the material characteristics of equipment in the production environment, providing an intuitive reference for subsequent production, manufacturing, maintenance and other links.

[0062] In addition to material editing, attribute binding is also an essential operation. Designers associate various relevant attribute information with the model. These attributes include equipment performance parameters, maintenance information, production capacity, etc. For example, for a machining center model, it is necessary to bind its spindle speed range, tool library capacity, machining accuracy and other performance parameters, as well as equipment maintenance cycle, maintenance points and other maintenance information. In this way, during the design process, designers can view and use this attribute information at any time to make a more reasonable layout design; in the subsequent production and maintenance stages, operators can also directly obtain key information from the model to improve production and maintenance efficiency.

[0063] In order to achieve a reasonable equipment layout, designers need to perform spatial operations such as rotation and translation on the imported model. Through rotation operations, the orientation of the equipment can be adjusted to meet the production process and operation requirements. For example, the processing direction of the machine tool is adjusted to be consistent with the material conveying direction to facilitate the processing and transfer of workpieces. Translation operations are used to determine the position of the equipment in three-dimensional space, and the equipment is placed in the best position considering factors such as the spacing between equipment, material handling channels, and personnel operating space. These spatial operations are a process of repeated adjustment and optimization. Designers need to comprehensively consider various factors to maximize production efficiency and optimize space utilization.

[0064] Design assistance functions (collision analysis, interface adsorption, measurement annotation, etc.)

[0065] Collision analysis: The collision analysis function plays a vital role in the equipment layout process. It can automatically detect whether there is interference between different equipment models. When the designer moves or adjusts the position of the equipment, the system calculates the spatial position relationship of the equipment in real time. Once the risk of collision between equipment is found, it will promptly issue an alarm to the designer. This helps to avoid the problem of collision between equipment and obstruction of production process in actual production, and ensure the rationality and safety of equipment layout.

[0066] Interface adsorption: The interface adsorption function facilitates accurate connection between models. On the production line, many devices need to be connected through specific interfaces, such as pipeline connection, conveyor belt and equipment docking, etc. When the designer brings two device models that need to be connected close together, the system will automatically adsorb them to the correct connection position according to the preset interface rules, ensuring the accuracy and stability of the interface and improving design efficiency and quality.

[0067] Measurement and annotation: The measurement and annotation function provides designers with a means to accurately obtain and record key information such as distances and angles between models. Designers can use this function to measure the distance between equipment, the distance between equipment and walls or other obstacles, and various angle information. At the same time, they can annotate these key information in the scene to form a clear design document. These annotation information not only helps the current design process, such as determining the rationality of equipment layout and planning the width of material handling channels, but also provides detailed reference for subsequent design review, construction and installation. Through these design-assisted functions, designers can complete the three-dimensional design plan of the production line more efficiently and accurately.

[0068] In some embodiments, the three-dimensional design end includes multiple design terminals; the three-dimensional design end is used to: according to the design instructions of each design terminal, retrieve the model from the model processing end through the data management end to create a three-dimensional production environment scene.

[0069] In an embodiment of the present invention, the VR review end includes multiple operation clients, and users can freely roam and review in the three-dimensional production environment scene in the VR environment. They can control their movement direction and speed in the scene through operating handles or other input devices just like walking in a real environment. During the roaming process, users can observe various aspects such as the layout of the equipment, the flow path of materials, and the operating space of personnel from different angles and distances. For example, the user can walk to the starting end of the production line and gradually observe the position and connection of each processing equipment along the direction of material transportation; or go deep into the narrow space between the equipment to check whether there is unreasonable space utilization or problems that may affect production operations. This free roaming review method allows users to discover some problems that are difficult to detect in traditional two-dimensional drawings or non-immersive three-dimensional views, such as blocked vision and narrow operating space.

[0070] The data query function is an important tool in the review process. Users can obtain detailed parameter information of the equipment through specific operations (such as selecting the equipment model in the VR environment and triggering the query command). These parameters include the model, specifications, performance indicators, production capacity, energy consumption, etc. of the equipment. For example, when a user queries a milling machine model, the system will display the key parameters of the milling machine, such as the spindle speed range, worktable size, and feed speed. In this way, users can check whether the equipment selection meets the production requirements, whether the equipment parameters match each other, and whether there is a waste of resources or insufficient performance. The data query function provides a way for the review work to gain an in-depth understanding of the characteristics of the equipment, which helps to discover potential problems in the design plan.

[0071] The information annotation function allows users to annotate problems found, key points that need attention, or other relevant information in the 3D production environment scene. Annotations can be text descriptions, graphic marks, or a combination of the two. For example, if a user finds that the installation location of a certain device may affect subsequent maintenance operations, he can mark "Insufficient maintenance space here, adjustment required" near the device; or when a certain section of the material conveying path is found to have a potential risk of blockage, the possible blockage location can be marked with a red line and the corresponding text explanation can be added. These annotation information not only facilitates the current user to record his own findings during the review process, but also facilitates other users participating in the review to understand the problem, and promotes communication and collaboration among team members. The annotation information can persist throughout the review process and can be seen by all users, becoming the basis for joint analysis and problem solving.

[0072] In addition to the review function, users can also use handheld devices (such as VR operating handles) to perform interactive design operations such as translation, rotation, and scaling of the three-dimensional production environment scene. These operations are implemented based on the interactive technology of VR devices. The buttons and joysticks on the operating handle are mapped to the interactive logic in the VR system. For example, the translation direction and speed of the scene can be controlled by moving the joystick, and the rotation or scaling of the scene can be achieved by pressing a specific button and combining the joystick operation. When the user performs these operations, the model in the scene will respond in real time, allowing the user to see the operation effect immediately. This interactive design operation method provides users with a convenient means of optimizing design.

[0073] Through the interactive design operation of the handheld device, users can optimize and improve the design plan. For example, when the equipment layout is found to be unreasonable, the user can move the equipment to a more suitable position through translation operation, and adjust the angle of the equipment through rotation operation to optimize the material flow path and personnel operation space. For some situations where the overall layout is too compact or loose, the relative spacing between the equipment can be adjusted through zooming operations to make the entire production environment more compact, efficient and in line with ergonomic principles. During the adjustment process, users can observe the collision between equipment, the smoothness of material transportation, and the changes in the operating space in real time, so as to quickly and accurately optimize the design plan, improve the rationality and scientificity of the production environment and equipment layout, and make the design plan more in line with the actual production needs.

[0074] In some embodiments, the operation client is further used to: display status information corresponding to each operation client in the three-dimensional production environment scene with different marks; and determine adjustment information of the three-dimensional production environment scene according to the status information corresponding to each operation client.

[0075] In an embodiment of the present invention, during the three-dimensional design and immersive interactive design review process, the operation end bears the important responsibility of collecting and transmitting user operation data. The operation end first packages and encapsulates all user operation data. The operation data here covers various actions and instruction information of users during the three-dimensional design and review process, including but not limited to the position change of the model in the three-dimensional space (such as translation, new coordinates after rotation), attribute modification (such as adjustment of device parameters, update of annotation content) and various interactive design operations (such as changes in zoom ratio, specific interactive actions performed through handheld devices), etc.

[0076] These operation data are encapsulated into UDP data frames. UDP (User Datagram Protocol) data frames are efficient and fast. They do not need to establish complex connections and ensure reliable data transmission like TCP. This feature is very suitable for data transmission scenarios with extremely high real-time requirements. Because in the design review process, the timeliness of data is more critical than the absolute accuracy of data. A small amount of data loss or error can be compensated by subsequent update mechanisms, but if the data transmission delay is too high, it will seriously affect the user experience and the coordination of design review.

[0077] The operation end sends the UDP data frame to the data management collaboration end. This sending process is realized through network connection. In a good network environment, data can be quickly transmitted from the operation end to the data management collaboration end, laying the foundation for subsequent real-time data processing and synchronization.

[0078] After receiving the UDP data frame from the operation end, the data management collaboration end starts to perform real-time data analysis. This is a complex and critical process. The data management collaboration end needs to accurately extract the operation information of each user from the data frame.

[0079] For data on model position changes, it needs to parse out the new coordinate values ​​of the model in the three-dimensional coordinate system as well as the direction and magnitude of the change. For example, if a device model is translated a certain distance by the user, the data management collaboration end must accurately obtain the translation vector, including the movement components on the x, y, and z axes. For attribute modification information, it is necessary to identify which device's attributes have been modified, as well as the new values ​​after the modification. For example, when a user adjusts the machining accuracy parameters of a machine tool, the data management collaboration end must clearly identify the device identification and new parameter values ​​involved in this modification operation. For various interactive design operation information, it is necessary to understand the type of operation (such as rotation, scaling, etc.) and the specific parameters of the operation (such as rotation angle, scaling ratio).

[0080] Through this precise data analysis, the data management collaboration end can fully grasp the operation content of each user in the 3D design end and the immersive interactive review end, and provide accurate data basis for subsequent status updates.

[0081] Based on the operation information obtained through analysis, the data management collaboration end comprehensively updates the scene status of the 3D design end and the immersive interactive review end. This update involves all-factor data such as scenario models, attributes, and location information.

[0082] For scene models, when users move, rotate, or scale the models, the data management collaboration end will change the model's position, direction, and size information in the scene accordingly. If the user adds or deletes a model, the data management collaboration end will also promptly reflect this change in the scene. In terms of attributes, any modification of device model attributes will be updated and recorded on the data management collaboration end. For example, changes in the performance parameters of the equipment, the addition of new annotation information, etc. will be accurately reflected in the system. For location information, whether it is the position adjustment of a single model or the change in the relative position relationship between a group of models, it can be properly handled.

[0083] This full-factor data update ensures that the entire 3D design and review scene always remains up-to-date and consistent with user operations, avoiding design confusion and misunderstandings caused by inconsistent data.

[0084] After completing the status update, the data management collaboration terminal broadcasts the updated status data to other operation clients. This broadcast process is the key link to achieve real-time information synchronization among multiple terminals and multiple professionals.

[0085] The broadcasted status data includes all updated full-factor data. After receiving this data, each operating client will immediately update the scene content displayed on its own 3D design end and immersive interactive review end. In this way, whether it is a designer who is doing 3D design or different users participating in the review in the VR environment, they can see the latest changes in the scene at the same time.

[0086] For example, when a user adjusts the position of a device and adds annotation information on the VR review client, all other operating clients (including clients of other VR review users and clients that are performing 3D design) will quickly receive the updated data and display the change in device position and new annotation content on their own interfaces. This real-time information synchronization ensures the smooth progress of online interactive design, allowing multiple professionals to work together in a unified, real-time updated environment to jointly review and optimize design plans, improve design efficiency and quality, and avoid design conflicts and duplication of work caused by data asynchrony.

[0087] In an embodiment of the present invention, after each operation client completes the adjustment, the three-dimensional production environment scene is not immediately generated, but status information is generated and added to the three-dimensional production environment scene displayed by the VR review terminal, and displayed in the corresponding adjustment position with controls of different colors / shapes (to distinguish the modifications generated by different operation clients), and other users can query the adjustment through the operation client for text comments and ratings. A superior operation client can be set to review the modifications of each client, or the adjustment plan with the highest summary value can be selected in a summary scoring manner. When the optimal adjustment plan is obtained, it is transmitted back to the data management terminal to adjust the stored three-dimensional model, that is, the three-dimensional production environment scene. In this way, multiple users can make adjustments at the same time without interfering with each other, and different users can also view the adjustments of other users at any time, ensuring data sharing and reasonable collaboration between different tasks.

[0088] In some embodiments, the data management end is further used to: update the three-dimensional production environment scene according to the adjustment information, and simultaneously update the 2D drawings corresponding to the three-dimensional production environment scene.

[0089] In the embodiment of the present invention, after the production line design process is completed, generating a two-dimensional drawing is one of the important contents of the design scheme export. The two-dimensional drawing is of key significance to the production and manufacturing process, and it provides detailed and accurate design information for production personnel.

[0090] When generating 2D drawings, the system extracts relevant data from the 3D production environment scene. The first is dimensioning. The system accurately marks the dimensions of the equipment, including length, width, height, diameter, etc. These dimension information are crucial for the manufacture and installation of the equipment. For example, for a machining center, the dimensions of its workbench, the travel range of each coordinate axis, the dimensions of the tool library, etc. will be marked to ensure that manufacturing workers can produce according to accurate specifications.

[0091] At the same time, the process requirements will also be reflected in detail in the two-dimensional drawings. This includes surface roughness requirements, tolerance range, assembly requirements, etc. For example, for parts that require high-precision matching, their matching tolerances will be clearly marked, such as the matching tolerances between shafts and holes, to ensure the assembly accuracy between parts. For some special processing techniques, such as heat treatment requirements and surface treatment requirements, they will also be noted in the drawings. For example, some parts need to be quenched to increase hardness, and the quenching temperature, medium and other parameters will be marked on the drawings.

[0092] In addition, the 2D drawings will also contain some other necessary information, such as part numbers, bills of materials, etc. Part numbers facilitate the identification and management of each component, and the bill of materials specifies the materials used for each component, including the model and specifications of the material, which helps the procurement department to accurately purchase the required materials. The generation of these 2D drawings meets the requirements of common design software. For example, they can be exported to common 2D drawing formats such as DWG (AutoCAD format) and DXF, so that other participants can view, edit and print them using various CAD software.

[0093] In addition to 2D drawings, 3D scene export is also an important part of design solution export. 3D scene files can provide other participants (such as construction and installation teams, customers, etc.) with an intuitive way to present design solutions.

[0094] When exporting a 3D scene, the system will retain all models, their attributes, and the spatial relationships between them in the 3D production environment scene. The model's geometry, material information, etc. are all fully included in the exported 3D scene file. For example, the complex shape, surface texture, color, and other visual effects of the equipment model can be accurately presented in the exported 3D scene, allowing viewers to clearly understand the appearance characteristics of the equipment.

[0095] The model's attribute information, such as the equipment's performance parameters and maintenance information, will also be exported with the model. This allows the construction and installation team to obtain relevant information about the equipment during the installation process to ensure the correctness of the installation. For customers, they can understand the functions and performance characteristics of the equipment by viewing the 3D scene and better evaluate whether the design solution meets their needs.

[0096] The exported 3D scene file formats are usually some common 3D file formats, such as OBJ, FBX, etc. These formats are compatible with most 3D design software and viewing software. In this way, other participants can easily open and view the 3D scene files in their own software environment for further analysis and collaborative applications, such as the formulation of construction and installation plans, the display and review of design plans, etc., to ensure that all links from design to production implementation can be smoothly connected and work together.

[0097] In addition, deep learning models can be trained and stored on the data management collaboration end to assist AR review. Deep learning models can be trained in the following ways:

[0098] First, obtain samples of 3D models containing correct and incorrect situations from various sources. These sources can be historical archives of actual design projects, test models with known error types generated by specific design software, etc. Make sure that the collected models cover a variety of possible error types, such as geometric errors (such as missing faces, overlaps, etc.), dimensional deviations, topological errors, etc.

[0099] Then, the collected 3D models are manually annotated to clearly indicate the type and location of errors in each model. This is the key to the subsequent training of deep learning models, because the model needs to learn to identify error features based on these annotations. Annotation can be done in many ways, such as marking specific areas of the model and attaching corresponding error descriptions, or giving the entire model an error code to represent its main error type.

[0100] Convert the 3D model into a format suitable for deep learning algorithm processing. A common approach is to mesh the 3D model, represent it as a set of vertices, edges, and faces, and normalize the data, such as unifying the coordinate range and normalizing the vertex coordinates. It may also be necessary to perform data enhancement operations on the model, such as rotation, translation, scaling, and other transformations, to increase the diversity of training data and improve the generalization ability of the model.

[0101] Next, divide the data into training set, validation set and test set. Usually, the preprocessed 3D model data is divided into training set, validation set and test set according to a certain ratio (such as the common 8:1:1). The training set is used to train the model, the validation set is used to adjust the model's hyperparameters (such as learning rate, number of layers, etc.) during the training process to optimize the model performance, and the test set is used to finally evaluate the performance of the trained model on unseen data.

[0102] Finally, the training set data is input into the selected deep learning model, and iterative training is performed according to the set learning rate, batch size and other parameters. In each iteration, the model predicts based on the input data, and then calculates the error between the predicted result and the true label through the loss function. The error is then back-propagated to each layer of the model using the back-propagation algorithm, and the model parameters are updated, so that the model's prediction ability is gradually improved. When the model training is completed, the test set data is input into the trained model for prediction, and then various evaluation indicators are calculated based on the prediction results and the true label, such as accuracy, recall, F1 value (applicable to multi-classification tasks), root mean square error (applicable to regression tasks), etc.

[0103] After training the deep learning model, the stored 3D production environment scene can be identified to find design errors, and the found design errors can be marked and displayed in each operation client of the VR review end. The operators of each operation client can directly view the content marked by the deep learning model in the 3D production environment scene for manual review. This not only reduces the tediousness of manual review through deep learning assistance, but also ensures accuracy through manual review, which is highly efficient. In addition, after each manual review, the training set can be updated according to the results of the manual review to ensure the accuracy of deep learning recognition.

[0104] The VR review terminal supports users to conduct production line roaming review from an immersive perspective, query equipment parameters, mark problems, and support interactive design operations such as equipment translation, rotation, and zoom. The corresponding scene operations and data changes can be synchronized to the 3D design terminal and VR terminal users in real time, realizing online collaboration among various personnel.

[0105] The 3D design end conducts forward design of the production line based on the 3D model, which can quickly discover errors, omissions, and defects. At the same time, designers can mark problems to form a problem list, and other personnel can quickly jump to the corresponding perspective through the problem list to locate and solve the problem.

[0106] Since three-dimensional design is relatively complex and has various forms, especially for large three-dimensional scenes, its complex structure and equipment composition have caused certain obstacles to the recognition of deep learning. Therefore, the present invention makes further improvements and designs a two-level deep learning model. The two-level deep learning model includes an upper-level deep learning model and multiple lower-level deep learning models.

[0107] Each subordinate deep learning model corresponds to a 3D design end, and the subordinate deep learning model will detect design errors in the corresponding 3D design end, specifically, errors in model import, material editing, attribute binding, rotation, translation, etc. At the same time, each subordinate deep learning model will extract the 3D design profile of the corresponding 3D design end, and send it to the superior deep learning model in combination with the detected design errors. The superior deep learning model detects connection errors in the overall 3D production environment scene based on the 3D design profiles and design errors of each 3D design end, specifically model collision errors, interface adsorption errors, and measurement annotation errors.

[0108] After the above improvements, each lower-level deep learning model can focus on identifying the local scenes designed by each three-dimensional design end, while the upper-level deep learning model is responsible for the connection detection between local scenes, which can greatly reduce the complexity of model recognition and improve recognition efficiency.

[0109] Figure 2 1 is a flowchart of the implementation of the production environment and equipment layout interactive immersive design method provided by the embodiment of the present invention. Figure 2 As shown in the figure, the interactive immersive design method of production environment and equipment layout includes:

[0110] S210, completing modeling, standardization processing and model management of production line equipment according to the model processing end;

[0111] S220, in response to a design instruction of a designer in the 3D design end, calling a model from the model processing end through the data management end, and creating a 3D production environment scene in the 3D design end;

[0112] S230, reviewing and optimizing the three-dimensional production environment scene in response to the VR review end including the operation information of each operation client;

[0113] S240, updating the status information of the three-dimensional production environment scene in the data management terminal according to the operation information of each operation client, and broadcasting the status information to each operation client.

[0114] Step 1: Modeling of production line equipment: constructing the production line equipment model through modeling software such as Revit and 3DMAX;

[0115] Step 2: Lightweight and standardized processing of production line equipment models: Use the model processing module to import third-party models offline or in batches through the API interface, and use methods and technologies such as model geometry complexity simplification, model texture and material optimization, and data compression to lightweight the model, while ensuring the model's precision and reducing the impact of design computer performance and network bandwidth on the model's application effect. Perform standardized conversion processing on model formats, scales, etc., and define interfaces and rules such as model mutual exclusion, linking, and safe distances based on the model's business attributes.

[0116] Step 3. Model warehousing management: Classify and manage production line environment models, machining equipment models such as machining centers, saws, broaching machines, grinders, planers, boring machines, milling machines, drilling machines, and transmission equipment models such as conveyor belts, and provide model calling services for production line 3D design and immersive interactive design modules through data interfaces.

[0117] Step 4, 3D design of production line: Create a 3D production environment scene, and multi-professional designers conduct 3D layout design based on the same work scene. Perform model import, material editing, attribute binding, rotation, translation and other spatial operations, and complete the scheme design through collision analysis, interface adsorption, measurement annotation and other functions.

[0118] Step 5: Immersive interactive design review: Multiple users conduct roaming review, data query, and information annotation in immersive scenes based on VR hardware devices, and use handheld devices to perform interactive design operations such as translation, rotation, and zoom to complete the review and optimization of the plan.

[0119] Step 6. During the 3D design and immersive interactive design review process, the operation end packages all users' operation data into UDP data frames and sends them to the data management collaboration end. The management end performs real-time data analysis and status updates, and broadcasts the status data to other operation clients, thereby achieving unified management and hot update of all-factor data such as scenario models, attributes, and location information of the 3D design end and the immersive interactive review end, ensuring real-time information synchronization among multiple terminals and multiple professionals, and realizing online interactive design.

[0120] Step 7: Export the production line design plan: Generate and export 2D drawings and 3D scenes. The exported results meet the requirements of common design software and can be used by other participants for collaborative applications.

[0121] In summary, the beneficial effects of the present invention are as follows: the present invention has built a set of interactive immersive design system for production environment and equipment layout. The system is based on conventional VR equipment on the market, and has developed model management end, data management collaboration end, three-dimensional design end and immersive interactive design review end software, realizing the whole process management of production line design from multi-source heterogeneous model import, lightweight, standardized processing, to three-dimensional solution scene establishment, production line three-dimensional design, to production line immersive design, roaming, review, and standardized production line two-dimensional solution export. At the same time, the invention breaks through the model lightweight and model and data online hot update synchronization technology, realizing online collaborative design of multiple professions, multiple people, and multiple terminals (immersive interactive design review end and three-dimensional design end), which not only solves the shortcomings of conventional two-dimensional design, but also takes into account the requirements of collaborative design. At the same time, the immersive editing and review method reduces the professional requirements for the owner, effectively improving the efficiency of design interaction and the quality of design results.

[0122] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0123] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0124] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0125] In the embodiments provided by the present invention, it should be understood that the disclosed devices / controllers and methods can be implemented in other ways. For example, the device / controller embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0126] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0127] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0128] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention.

Claims

1. An interactive immersive design system for production environment and equipment layout, characterized in that: include: VR review end, 3D design end, model processing end and data management end; The VR review end, the three-dimensional design end, and the model processing end are respectively connected to the data management end; The model processing end is used to complete the modeling, standardization processing and model management of production line equipment; The three-dimensional design end is used to respond to the design instructions of the designer, retrieve the model from the model processing end through the data management end, and create a three-dimensional production environment scene; The VR review end includes a plurality of operation clients; each operation client reviews and optimizes the three-dimensional production environment scene in response to the operation information; The data management terminal is used to update the status information of the three-dimensional production environment scene according to the operation information of each operation client, and broadcast the status information to each operation client; The hardware part of the VR review end includes VR head display equipment, VR operating handles and VR basic platform; The operation client is used to: Roaming, data query, and information annotation of the three-dimensional production environment scene to complete the review process; Perform interactive design operations on the three-dimensional production environment scene to complete the optimization process; The data management end is provided with a two-level deep learning model; the two-level deep learning model includes an upper-level deep learning model and a plurality of lower-level deep learning models; wherein each lower-level deep learning model corresponds to a 3D design end, and the lower-level deep learning model detects design errors of the corresponding 3D design end; the upper-level deep learning model detects connection errors in the overall 3D production environment scene based on the 3D design profiles and design errors of each 3D design end; The two-level deep learning model is used to mark and display the errors found in the three-dimensional production environment scene in each operation client of the VR review end; The operation client is also used to manually review the errors marked by the two-level deep learning model, and update the training set of the two-level deep learning model in the data management end according to the manual review results.

2. The interactive immersive design system for production environment and equipment layout according to claim 1, characterized in that: The operation client is also used to: Displaying status information corresponding to each operation client in the three-dimensional production environment scene with different marks; According to the status information corresponding to each operation client, the adjustment information of the three-dimensional production environment scene is determined.

3. The interactive immersive design system for production environment and equipment layout according to claim 1, characterized in that: The data management terminal is also used for: According to the adjustment information, the three-dimensional production environment scene is updated, and the 2D drawing corresponding to the three-dimensional production environment scene is updated at the same time.

4. The interactive immersive design system for production environment and equipment layout according to claim 1, characterized in that: The three-dimensional design terminal is used for: In response to the design instruction of the designer, the model is retrieved from the model processing end through the data management end to complete any one or more of the following spatial operations: model import, material editing, attribute binding to rotate, translate; The completed model is subjected to collision analysis, interface adsorption, and measurement and annotation to obtain a three-dimensional production environment scene.

5. The interactive immersive design system for production environment and equipment layout according to claim 1, characterized in that: The three-dimensional design end includes a plurality of design terminals; the three-dimensional design end is used for: According to the design instructions of each design terminal, the data management end retrieves the model from the model processing end to create a three-dimensional production environment scene.

6. The interactive immersive design system for production environment and equipment layout according to claim 1, characterized in that: The model processing end is used for: Build models for each production line equipment; Import models of third-party devices; All models are lightweight and annotated.

7. The interactive immersive design system for production environment and equipment layout according to claim 1, characterized in that: The model processing end is used for: Classify and call each model.

8. The interactive immersive design system for production environment and equipment layout according to claim 1, characterized in that: The production line equipment includes machining equipment and transmission equipment.

9. An interactive immersive design method for production environment and equipment layout, characterized in that: The production environment and equipment layout interactive immersive design system applied to any one of claims 1 to 8 above; the method comprises: Complete the modeling, standardization and model management of production line equipment based on the model processing end; In response to a design instruction from a designer in the 3D design end, the data management end retrieves the model from the model processing end, and creates a 3D production environment scene in the 3D design end; The three-dimensional production environment scene is reviewed and optimized in response to the operation information of each operation client in the VR review terminal; wherein the hardware part of the VR review terminal includes a VR head display device, a VR operation handle and a VR basic platform; In response to the operation information of each operation client in the VR review terminal, the three-dimensional production environment scene is reviewed and optimized, including: Roaming, data query, and information annotation of the three-dimensional production environment scene to complete the review process; Perform interactive design operations on the three-dimensional production environment scene to complete the optimization process; According to the operation information of each operation client, the state information of the three-dimensional production environment scene in the data management terminal is updated, and the state information is broadcast to each operation client; The data management end is provided with a two-level deep learning model; the two-level deep learning model includes an upper-level deep learning model and a plurality of lower-level deep learning models; wherein each lower-level deep learning model corresponds to a three-dimensional design end; The method further comprises: Detect design errors on the corresponding 3D design side based on the lower-level deep learning model; Adopting the superior deep learning model, based on the 3D design profiles and design errors of each 3D design end, it detects the connection errors in the overall 3D production environment scene; The errors found in the 3D production environment scene will be marked and displayed in each operation client of the VR review end; Obtain the manual review results of the operation client, and update the training set of the two-level deep learning model in the data management end according to the manual review results of the operation client.

Images