Intelligent interaction control system and method based on three-dimensional engine

Abstract

The invention relates to the technical field of smart parks, and discloses an intelligent interaction control system and method based on a three-dimensional engine, and the method comprises the steps: judging whether a cross-region object exists or not based on a boundary image set, obtaining a target image set corresponding to the cross-region object from the boundary image set when the cross-region object exists, constructing a corresponding target three-dimensional model according to the target image set, responding to a trigger operation acting on the interaction control, generating a target frame block subset, decoding a target model frame block to obtain a shared model frame block, and displaying the target three-dimensional model in an interaction interface according to the shared model frame block, according to the method, the target three-dimensional model corresponding to the cross-regional object can be generated by integrating the image data of each monitoring sub-region, and the network data transmission quantity and the consumption of hardware resources are reduced in the process of displaying the target three-dimensional model, so that the target three-dimensional model can be displayed in time.

Description

Technical Field

[0001] The present invention relates to the field of smart park technology, and more specifically, to an intelligent interactive control system and method based on a three-dimensional engine. Background Art

[0002] With the rapid development of the Internet of Things, big data, blockchain and artificial intelligence technologies, the construction of smart parks has become an important direction of digital transformation. Traditional park management systems are usually limited to static data display and management functions, and cannot meet the park's needs for dynamic perception, real-time interaction and intelligent decision-making. In addition, due to the existence of data islands, data between different systems is difficult to effectively integrate, resulting in low overall efficiency of park management. Therefore, there is an urgent need for an intelligent system that integrates multiple emerging technologies to break down data barriers and improve the level of intelligence in park management.

[0003] In the prior art, the Chinese patent with publication number CN110225296A provides a method, system and device for intelligent monitoring of three-dimensional real scenes in a park. The invention associates monitoring video streams, park equipment information and environmental information with the three-dimensional model of the park and displays them visually, so that park managers can intuitively monitor the real scene and equipment status on the three-dimensional model. The Chinese patent with publication number CN117082205A provides a smart park security monitoring and management system, which realizes intelligent monitoring and efficient processing and display of security events through wireless sensors, cameras, three-dimensional models and visualization platforms.

[0004] Although existing technologies have achieved real-time monitoring and visual display of the park, in actual applications, due to the large area of ​​the park, it is difficult to share monitoring data between areas. When a device spans multiple areas at the same time, if the device fails, the system will find it difficult to perform three-dimensional display in a timely manner.

[0005] In view of this, the present invention proposes an intelligent interactive control system and method based on a three-dimensional engine to solve the above problems. Summary of the invention

[0006] In order to overcome the above-mentioned defects of the prior art, the present invention provides an intelligent interactive control system and method based on a three-dimensional engine.

[0007] To achieve the above object, the present invention provides the following technical solutions:

[0008] In a first aspect, an intelligent interactive control method based on a three-dimensional engine is applied to an interactive interface, and interactive controls are displayed through the interactive interface, including:

[0009] Obtain image data sets of H monitoring sub-areas, process the image data sets to determine a boundary image set, and determine whether there is a cross-area object based on the boundary image set, where H is an integer greater than 1;

[0010] When there is a cross-region object, a target image set corresponding to the cross-region object is obtained from the boundary image set, and a corresponding target three-dimensional model is constructed according to the target image set;

[0011] In response to a trigger operation acting on the interactive control, a target frame block subset is generated, wherein the target frame block subset includes at least one target model frame block;

[0012] The target model frame block is decoded to obtain a shared model frame block, and the target three-dimensional model is displayed in the interactive interface according to the shared model frame block.

[0013] Furthermore, the method for obtaining the image data set includes:

[0014] Obtain a planning map corresponding to H monitoring sub-areas, determine the boundary lines between the H monitoring sub-areas according to the planning map, determine the initial boundary area based on the boundary lines and a preset standard distance, obtain all image data of the initial boundary area, and generate an image data set, wherein the standard distance is the distance in the vertical direction of the boundary line, and the image data set includes H initial boundary images.

[0015] Furthermore, the method of processing the image data set to determine the boundary image set includes:

[0016] Based on the edge detection algorithm, the boundary lines between each monitoring sub-area are identified, and the image data set is cropped according to the preset standard distance through the image segmentation algorithm to obtain H area cropped images. The H area cropped images are filtered based on the edge-preserving smoothing algorithm to obtain a boundary image set.

[0017] Furthermore, the method for determining whether there is a cross-region object based on the boundary image set includes:

[0018] Input the boundary image set into the pre-built object detection model to obtain the detection results, including cross-region and non-cross-region results;

[0019] The object detection model construction method includes:

[0020] Obtain Q sets of training data, where Q is a positive integer greater than 1, and the training data includes a historical boundary image set and historical detection results;

[0021] The historical boundary image set and historical detection results are used as sample sets, the sample sets are divided into training sets and test sets, and a classifier is constructed;

[0022] The historical boundary image set in the training set is used as input data, and the historical detection results in the training set are used as output data to train the classifier to obtain an initial classifier;

[0023] The initial classifier is tested using the test set, and the classifier that meets the preset accuracy is output as the object detection model.

[0024] Furthermore, the method for obtaining a target image set corresponding to a cross-region object includes:

[0025] Identify cross-regional objects, obtain unique identification information corresponding to the cross-regional objects, obtain the corresponding standard image set based on the unique identification information, determine the corresponding shooting device information based on the standard image set, mobilize the corresponding shooting device based on the shooting device information, obtain the corresponding real-time image set and unify the format, and use the real-time image set in the unified format as the target image set.

[0026] Furthermore, the method of constructing a corresponding target three-dimensional model according to the target image set includes:

[0027] The target image set is identified to obtain the device model, a pre-built initial three-dimensional model is determined based on the device model, and a corresponding target three-dimensional model is constructed according to the target image set, simulation analysis software and the initial three-dimensional model, wherein the initial three-dimensional model at least includes a structural three-dimensional model.

[0028] Furthermore, the method of constructing the corresponding target three-dimensional model includes:

[0029] Structural feature data is obtained based on the target image set, and the structural feature data is input into the simulation analysis software. Then, the initial three-dimensional model is simulated and corrected according to the simulation analysis software to construct the corresponding target three-dimensional model.

[0030] Further, the method of generating a target frame block subset includes:

[0031] In response to a first trigger operation applied to the interactive control, a first model frame block set is output according to the first trigger operation, in response to a second trigger operation and obtaining a corresponding second model frame block set, a target frame block subset is generated based on the first model frame block set and the second model frame block set.

[0032] In a second aspect, an intelligent interactive control system based on a three-dimensional engine is used to implement the above-mentioned intelligent interactive control method based on a three-dimensional engine, comprising:

[0033] The first processing module is used to obtain image data sets of H monitoring sub-areas, process the image data sets to determine boundary image sets, and determine whether there is a cross-area object based on the boundary image sets, where H is an integer greater than 1;

[0034] Model building module: when there is a cross-region object, it is used to obtain a target image set corresponding to the cross-region object from the boundary image set, and to build a corresponding target 3D model according to the target image set;

[0035] A second processing module: used for generating a target frame block subset in response to a trigger operation acting on the interactive control, wherein the target frame block subset includes at least one target model frame block;

[0036] Interaction module: used to decode the target model frame block to obtain the shared model frame block, and display the target three-dimensional model in the interaction interface according to the shared model frame block.

[0037] In a third aspect, a computer-readable storage medium stores a computer program, and when the computer program is executed, the intelligent interactive control method based on a three-dimensional engine is implemented.

[0038] Compared with the prior art, the present invention has the following beneficial effects:

[0039] The present invention first processes an image data set to determine a boundary image set, and determines whether there is a cross-regional object based on the boundary image set. When a cross-regional object exists, a target image set corresponding to the cross-regional object is obtained from the boundary image set, and a corresponding target three-dimensional model is constructed according to the target image set. Then, in response to a trigger operation acting on the interactive control, a target frame block subset is generated, the target model frame block is decoded to obtain a shared model frame block, and the target three-dimensional model is displayed in an interactive interface according to the shared model frame block. When a cross-regional object fails, the present invention can generate a target three-dimensional model corresponding to the cross-regional object by integrating image data of each monitoring sub-area, and reduce network data transmission volume and hardware resource consumption during the display of the target three-dimensional model, thereby ensuring that the target three-dimensional model can be displayed in a timely manner. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 It is a flow chart of an intelligent interactive control method based on a three-dimensional engine in the present invention;

[0041] Figure 2 It is a structural schematic diagram of an intelligent interactive control system based on a three-dimensional engine in the present invention;

[0042] Figure 3 It is a planning diagram of each monitoring sub-area in the present invention;

[0043] Figure 4 is a schematic diagram of the initial boundary area in the present invention;

[0044] Figure 5 A schematic diagram of a computer-readable storage medium in the present invention.

[0045] Reference numerals:

[0046] 10, park; 20, monitoring sub-area; 30, boundary line; 40, initial boundary area; SD, standard distance. DETAILED DESCRIPTION

[0047] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0048] Example 1

[0049] See also Figure 1 As shown, this embodiment provides an intelligent interactive control method based on a three-dimensional engine, which is applied to an interactive interface and displays interactive controls through the interactive interface, including:

[0050] S10: Obtain image data sets of H monitoring sub-areas, process the image data sets to determine a boundary image set, and determine whether there is a cross-area object based on the boundary image set, where H is an integer greater than 1;

[0051] In this embodiment, the monitoring sub-area can be any area in the park. The park usually has office areas, production and manufacturing areas, R&D areas, life support areas, logistics and warehousing areas, etc. It can be understood that during the construction process of the park, due to differences in construction speed or equipment models, the monitoring systems of various areas in the park are not unified. Therefore, the specifications or formats of the monitoring data (such as image data) obtained from each area are inconsistent, resulting in the inability to quickly integrate the monitoring data, affecting the subsequent three-dimensional display.

[0052] Methods for obtaining image data sets include:

[0053] Obtain a planning map corresponding to H monitoring sub-areas, determine the boundary lines between the H monitoring sub-areas according to the planning map, determine the initial boundary area based on the boundary lines and a preset standard distance, obtain all image data of the initial boundary area, and generate an image data set, wherein the standard distance is the distance in the vertical direction of the boundary line, and the image data set includes H initial boundary images.

[0054] It should be noted that the initial boundary image refers to the image of the initial boundary area. Since there are H monitoring sub-areas, H initial boundary images can be obtained through H shots. Each initial boundary image corresponds to a monitoring sub-area, such as Figure 3 and Figure 4 As shown, Figure 3 The park 10, the monitoring sub-area 20, and the boundary line 30 are shown. Figure 4 The initial boundary area 40 and the standard distance SD are shown, which are given by Figure 4 It can be seen that the standard distance SD is the distance in the vertical direction of the boundary line 30. By setting corresponding area dividing lines on both sides of the boundary line 30, the vertical distance between the boundary line 30 and the area dividing line is the standard distance SD, and the area between the two area dividing lines is the initial boundary area. It can be understood that the specific data of the standard distance SD can be determined by personnel in this field according to actual conditions, and the planning map mentioned above is pre-stored in the database. The image data of the initial boundary area can be the monitoring image of the area, and this embodiment will not go into too much detail.

[0055] In this embodiment, by determining the boundary lines and preset standard distances between monitoring sub-areas, the initial boundary area can be accurately delineated, so that the system can more effectively detect objects across multiple areas, ensuring the comprehensiveness and accuracy of monitoring. At the same time, only image data within the initial boundary area is obtained, avoiding the processing of data for the entire area, reducing the amount of data and calculation, and improving the efficiency of data processing and fusion. A clear initial boundary area also enables the system to more quickly identify and process monitoring data of cross-regional objects, thereby improving real-time monitoring and response speed. Since monitoring equipment in different areas of the park may have differences in specifications or formats, directly fusing all image data will face many compatibility issues. By first obtaining a planning map and determining a unified initial boundary area, the format consistency of image data in each sub-area after standardized processing can be ensured, making it easier to perform data fusion and integration later.

[0056] The method of processing the image data set to determine the boundary image set includes:

[0057] Based on the edge detection algorithm, the boundary lines between each monitoring sub-area are identified, and the image data set is cropped according to the preset standard distance through the image segmentation algorithm to obtain H area cropped images. The H area cropped images are filtered based on the edge-preserving smoothing algorithm to obtain a boundary image set.

[0058] It should be noted that, in this embodiment, the image data set includes H initial boundary images, and the initial boundary images do not completely correspond to the initial boundary areas. Usually, in order to ensure the integrity of the image, the shooting range should be appropriately expanded during the process of obtaining the initial boundary images, so that each initial boundary image not only contains the entire content of the initial boundary area, but also can capture a part of the adjacent area. This approach helps to ensure that the boundary lines can be identified more accurately during the image segmentation and edge detection process, and effectively determine whether there are cross-regional objects.

[0059] It should be added that the edge detection algorithm mentioned above can be a Canny algorithm, a Sobel algorithm, etc., the image segmentation algorithm can be region-based segmentation, cluster segmentation, threshold segmentation, etc., and the edge-preserving smoothing algorithm can be a bilateral filter or a guided filter. The above algorithms are all existing technologies and will not be described in detail in this embodiment.

[0060] In this embodiment, the cropped area image is processed using an edge-preserving smoothing algorithm, which can better preserve important edge information in the image while removing noise. This enables the system to be highly sensitive to changes in the boundary area and to promptly detect objects crossing the boundary line. Even if the movement or change of the object is small, it can be clearly identified. Moreover, by identifying and processing the boundary area in advance, the system only needs to process image data near the boundary, rather than all data in the entire monitoring area. This method reduces the amount of data that needs to be calculated and processed, reduces the computational complexity of the algorithm and hardware resource consumption, and enables the system to respond more quickly to the detection needs of cross-regional objects.

[0061] Methods for determining whether there is a cross-region object based on a boundary image set include:

[0062] The boundary image set is input into the pre-built object detection model to obtain the detection results, which include cross-region and non-cross-region.

[0063] The object detection model construction method includes:

[0064] Obtain Q sets of training data, where Q is a positive integer greater than 1, and the training data includes a historical boundary image set and historical detection results;

[0065] The historical boundary image set and historical detection results are used as sample sets, the sample sets are divided into training sets and test sets, and a classifier is constructed;

[0066] The historical boundary image set in the training set is used as input data, and the historical detection results in the training set are used as output data to train the classifier to obtain an initial classifier;

[0067] The initial classifier is tested using the test set, and a classifier that meets the preset accuracy is output as the object detection model. The classifier is preferably a naive Bayes model or a support vector machine model.

[0068] It can be understood that the criterion for whether an object crosses regions in this embodiment is that when the object crosses the boundary line and both ends of the object extend to the outside of the initial boundary area, the object is judged to be cross-regional, otherwise, it is judged to be non-cross-regional. The purpose of this is that when the three-dimensional model corresponding to the object is subsequently established, if the object crosses regions, the format of the target image set needs to be unified before subsequent modeling can be performed. If the object is only in the initial boundary area, there is no need for complex data format unification and cross-regional processing, because all relevant image data of the object already exists in a single monitoring sub-area. The advantage of this is that it simplifies the data processing process, reduces unnecessary computing resource consumption and data fusion steps, avoids unnecessary system complexity and computing overhead, and improves the efficiency and real-time performance of the system for three-dimensional display.

[0069] S20: when there is a cross-region object, obtaining a target image set corresponding to the cross-region object from the boundary image set, and constructing a corresponding target three-dimensional model according to the target image set;

[0070] In this embodiment, the target image set refers to the structural image set of a cross-regional object. An exemplary cross-regional object may be a transportation pipeline in a park. The structural image set may be multiple perspective images of the pipeline in each monitoring sub-region, images of the connection points or interface parts of the pipeline when crossing the sub-region boundary, and endoscopic images of the inside of the pipeline.

[0071] The method for obtaining a target image set corresponding to a cross-region object includes:

[0072] Identify cross-regional objects, obtain unique identification information corresponding to the cross-regional objects, obtain the corresponding standard image set based on the unique identification information, determine the corresponding shooting device information based on the standard image set, mobilize the corresponding shooting device based on the shooting device information, obtain the corresponding real-time image set and unify the format, and use the real-time image set in the unified format as the target image set.

[0073] The above-mentioned unique identification information can be an object ID number or a geometric feature fingerprint. It can be understood that the geometric feature fingerprint refers to the unique geometric shape, texture or surface feature of a cross-regional object. Taking the cross-regional object as a conveying pipeline as an example, the geometric feature fingerprint can be the outer diameter, shape, color, surface texture, etc. of the conveying pipeline. Identifying cross-regional objects means that feature recognition is performed on cross-regional objects in a boundary image set to obtain the corresponding geometric feature fingerprint. Identifying cross-regional objects can be achieved using existing computer vision algorithms. The computer vision algorithm can be a feature point detection algorithm (such as S IFT, SURF, ORB, etc.) to extract key feature points in the image. These feature points can describe the geometric shape, texture or surface features of the object.

[0074] Among them, the above-mentioned standard image set refers to the reference images taken from different angles, positions and specific structural parts after the installation of cross-regional objects is completed, and is pre-stored in the database. However, these standard images only record the initial state of the object during installation, including the standard viewing angle and structural features at a specific time point, and cannot reflect the changes or dynamic information of the object in a real-time environment.

[0075] Therefore, the purpose of acquiring a real-time image collection is to ensure that subsequent 3D modeling can be based on actual conditions, so that the 3D display can more accurately reflect the state of cross-regional objects in the current environment. The real-time image collection can capture the actual geometric features and physical properties of the object, including its current appearance, position and structure, thereby significantly improving the accuracy and reliability of the 3D model.

[0076] In contrast, real-time image collections can better reflect the real-time changes and dynamic characteristics of objects, allowing the three-dimensional display to be updated at any time to promptly reflect the latest status of the object. This dynamic display can help managers and decision makers grasp the current situation more comprehensively and accurately, thereby making timely and effective decisions.

[0077] It should be added that, since different monitoring sub-areas may be equipped with camera devices of different brands, models or configurations, these devices may differ in image resolution, frame rate, color depth, viewing angle, focal length, etc., resulting in inconsistent image formats and quality. Different monitoring sub-areas may have changes in environmental factors such as lighting conditions, weather conditions, and obstructions. These factors will affect the brightness, contrast, color and clarity of the image, resulting in differences in the visual performance of the real-time image set. Alternatively, the image acquisition system and data processing system of each monitoring sub-area may use different hardware devices such as image acquisition cards, data transmission interfaces, and processors. The differences in the performance and technical specifications of these devices will also lead to differences in image data formats. Therefore, in this embodiment, it is necessary to unify the format of the real-time image set. The advantage of doing so is that it ensures that subsequent processing steps (such as three-dimensional modeling, image fusion, etc.) can proceed smoothly, avoiding errors or information loss caused by incompatible data formats, and using image data in a unified format can reduce modeling errors caused by inconsistent formats, ensuring the geometric accuracy and physical consistency of the model, which helps to build a more accurate three-dimensional model that truly reflects the state of the object in the actual environment.

[0078] The method for constructing a corresponding target three-dimensional model according to a target image set includes:

[0079] The target image set is identified to obtain the device model, a pre-built initial three-dimensional model is determined based on the device model, and a corresponding target three-dimensional model is constructed according to the target image set, simulation analysis software and the initial three-dimensional model, wherein the initial three-dimensional model at least includes a structural three-dimensional model.

[0080] It should be noted that the initial three-dimensional model is a mechanical structure model. For example, when the cross-regional object is a conveying pipeline, the initial three-dimensional model is the mechanical structure model of the conveying pipeline. It can be understood that the mechanical structure model of the conveying pipeline is pre-built through detailed design drawings and technical specifications of the conveying pipeline, including material properties, geometric dimensions, etc., and then through three-dimensional software such as SolidWorks. The above-mentioned equipment models usually correspond one-to-one to the corresponding cross-regional objects, so the corresponding initial three-dimensional model can be directly determined by the equipment model.

[0081] The method of constructing the corresponding target three-dimensional model includes:

[0082] Structural feature data is obtained based on the target image set, and the structural feature data is input into the simulation analysis software. Then, the initial three-dimensional model is simulated and corrected according to the simulation analysis software to construct the corresponding target three-dimensional model.

[0083] Among them, the structural feature data includes but is not limited to geometric shape information, dimensional data and structural integrity information, etc. The geometric shape information refers to the three-dimensional geometric shape description of the object, including information such as length, width, height, curvature, bending, etc. The dimensional data refers to the specific dimensional parameters of the object, such as outer diameter, inner diameter, thickness, cross-sectional area, etc. The structural integrity information refers to the location information of cracks or damages. The structural feature data characterizes the real-time status of the cross-regional object. For example, if the cross-regional object is a conveying pipeline, the structural integrity information may be the crack location information and damage location information on the inner and outer surfaces of the conveying pipeline. Then, when the structural feature data is input into the simulation analysis software, the simulation analysis software will use these structural feature data to simulate and correct the initial three-dimensional model to reflect the actual status of the cross-regional object. Specifically:

[0084] The simulation analysis software will first correct the shape of the initial three-dimensional model based on the geometric shape information (such as length, width, height, curvature, bending, etc.) to ensure that the model is consistent with the actual geometric shape of the target object. This involves redefining or adjusting the model's vertex coordinates, edges, and facets.

[0085] The software will use the dimensional data (such as outer diameter, inner diameter, thickness, cross-sectional area, etc.) to adjust the various parts of the model to conform to the specific dimensional parameters of the actual object. This process ensures that the proportions of the model in all dimensions are consistent with the real object.

[0086] The simulation analysis software will use the structural integrity information (such as the location of cracks and damage) to mark the corresponding defects or damaged areas on the 3D model. The software will simulate the impact of these cracks or damage on the overall structural performance of the object.

[0087] It should be added that the acquisition of structural feature data based on the target image set mentioned above can be achieved through SIFT algorithm, threshold segmentation algorithm, etc. The SIFT algorithm is used to identify the geometric shape features of the object, and the threshold segmentation algorithm is used to perform threshold processing on the image according to the difference in pixel intensity to extract the area of ​​interest (such as cracks, damaged areas, etc.). The simulation analysis software can be ANSYS, SolidWorks, ABAQUS and other software. The adjustment of the model by the simulation analysis software is a prior art, and this embodiment will not go into details.

[0088] In this embodiment, by identifying cross-regional objects and acquiring real-time image sets, the system can capture the real-time status of objects in the current environment, and then input the structural feature data under these real-time states into the simulation analysis software to dynamically correct and update the initial three-dimensional model. The advantage of this is that the three-dimensional model can reflect the actual geometric characteristics, physical properties and dynamic changes of the object in real time, so that the model has higher accuracy and real-time performance. By uniformly formatting the real-time collected image data and correcting and fusing it with the initial model in the simulation analysis software, it can be ensured that the final generated three-dimensional model is consistent with the environmental status of the actual object. This not only improves the coordination and compatibility of the model between various sub-regions, but also reduces model deviations caused by equipment differences or inconsistent data formats.

[0089] S30: generating a target frame block subset in response to a trigger operation acting on the interactive control, wherein the target frame block subset includes at least one target model frame block;

[0090] In this embodiment, the interactive control refers to a three-dimensional display control. The staff can use the interactive control to rotate the target three-dimensional model. Each side of the target three-dimensional model can be clearly displayed, thereby improving the visualization effect of the target three-dimensional model. The model frame block refers to a single-frame image data block sampled or generated at a certain time interval during the three-dimensional display process of the target three-dimensional model. These frame blocks represent the appearance and details of the three-dimensional model at a specific viewing angle and state. It can be understood that there are several model frame blocks on each side of the target three-dimensional model.

[0091] The method of generating a target frame block subset includes:

[0092] In response to a first trigger operation applied to the interactive control, a first model frame block set is output according to the first trigger operation, in response to a second trigger operation and obtaining a corresponding second model frame block set, a target frame block subset is generated based on the first model frame block set and the second model frame block set.

[0093] It should be noted that the first trigger operation refers to the initial interactive operation performed by the staff through the three-dimensional display control, for example, the user clicks and drags the three-dimensional display interface to make the target three-dimensional model start to rotate to one side and then stop, and the second trigger operation refers to the additional interactive operation performed by the user with the three-dimensional display control after the initial display or transformation, for example, the user drags again to further rotate the three-dimensional model to make the target three-dimensional model rotate to the other side and then stop. However, it is worth noting that the one side and the other side mentioned above must be adjacent sides. Adjacent sides refer to two sides on the target three-dimensional model that share a common edge or common vertex. In three-dimensional space, these sides are geometrically closely connected, and the relative positions of adjacent sides are in direct contact with each other and have a physical connection relationship. The purpose of this is to ensure the continuity and integrity of the three-dimensional display, so that users can clearly view the details of the target model from different perspectives.

[0094] Among them, the target frame block subset refers to a part of the frame blocks extracted from the set of all frame blocks of the three-dimensional model after responding to the user's interactive operation, which is used to continuously display the image data of the specific perspective and state of adjacent faces. The target model frame block refers to the model frame block common to two adjacent faces of the target three-dimensional model. The target model frame block is used to connect and transition the visual display between adjacent faces to ensure that the display of the three-dimensional model maintains visual continuity and consistency between adjacent faces. The target model frame block contains multiple perspectives that gradually change from one face to an adjacent face. It helps generate a series of continuous image frame blocks, allowing users to naturally transition from one face to another while maintaining the overall coherence and detail clarity of the model. When the user rotates or switches the perspective to view different faces of the target three-dimensional model, the target model frame block is used to display the transition area between two adjacent faces to ensure a smooth transition during the perspective change. The purpose of this is to avoid sudden breaks or jumps during display and improve the fluency of the visualization effect.

[0095] S40: Decode the target model frame block to obtain a shared model frame block, and display the target three-dimensional model in the interactive interface according to the shared model frame block.

[0096] In this embodiment, since the target three-dimensional models in this embodiment are all displayed by the interactive interface, but the interaction for fault monitoring requires real-time performance, the target three-dimensional models in this embodiment are all generated and displayed in real time, so the target three-dimensional models are all transmitted in real time, but due to reasons such as network speed and display memory, delays, freezes or inability to perform high-quality rendering are prone to occur during the display of the target three-dimensional model. Therefore, this embodiment first displays some model frame blocks, that is, displays the first model frame block set in the first perspective. When the staff switches to the second perspective, the target model frame blocks are first obtained, decoded and pre-stored. In this way, in the process of the next step of transmission, there is no need to transmit the target model frame blocks, and only the pre-stored target model frame blocks need to be displayed.

[0097] The target model frame block can be decoded by a corresponding software decoder, such as FFmpeg or DirectX Video Accession.

[0098] It should be added that, in the process of displaying the target three-dimensional model, this embodiment does not only rely on the shared model frame blocks for display, but also displays it in combination with the frame blocks corresponding to the model surface to be displayed.

[0099] Among them, the combination of the above three steps helps to reduce the amount of network data transmission and the consumption of hardware resources. In step S20, by acquiring and constructing a target three-dimensional model of a cross-regional object, global scanning and data transmission of the entire area can be avoided. In step S30, by only generating a subset of target frame blocks, unnecessary frame block data transmission can be avoided. In step S40, by pre-storing shared model frame blocks, repeated transmission and decoding of the same data blocks are avoided. Combined, this strategy reduces the data transmission requirements, saves network bandwidth and memory usage of display devices, and improves the efficiency of the overall system.

[0100] In this embodiment, the image data set is first processed to determine the boundary image set, and it is determined whether there is a cross-regional object based on the boundary image set. When there is a cross-regional object, the target image set corresponding to the cross-regional object is obtained from the boundary image set, and the corresponding target three-dimensional model is constructed according to the target image set. Then, in response to the trigger operation acting on the interactive control, a target frame block subset is generated, the target model frame block is decoded to obtain a shared model frame block, and the target three-dimensional model is displayed in the interactive interface according to the shared model frame block. When a cross-regional object fails, this embodiment can generate a target three-dimensional model corresponding to the cross-regional object by integrating the image data of each monitoring sub-area, and reduce the network data transmission volume and the consumption of hardware resources during the display of the target three-dimensional model, thereby ensuring that the target three-dimensional model can be displayed in a timely manner.

[0101] Example 2

[0102] See also Figure 2 As shown, based on the same inventive concept, this embodiment provides an intelligent interactive control system based on a three-dimensional engine. For details not provided in this embodiment, please refer to the description of the relevant parts in Embodiment 1. The system includes:

[0103] See also Figure 1 As shown, this embodiment discloses a three-dimensional engine-based intelligent interactive control method, which is applied to an interactive interface and displays interactive controls through the interactive interface, including:

[0104] The first processing module is used to obtain image data sets of H monitoring sub-areas, process the image data sets to determine boundary image sets, and determine whether there is a cross-area object based on the boundary image sets, where H is an integer greater than 1;

[0105] In this embodiment, the monitoring sub-area can be any area in the park. The park usually has office areas, production and manufacturing areas, R&D areas, life support areas, logistics and warehousing areas, etc. It can be understood that during the construction process of the park, due to differences in construction speed or equipment models, the monitoring systems of various areas in the park are not unified. Therefore, the specifications or formats of the monitoring data (such as image data) obtained from each area are inconsistent, resulting in the inability to quickly integrate the monitoring data, affecting the subsequent three-dimensional display.

[0106] Methods for obtaining image data sets include:

[0107] Obtain a planning map corresponding to H monitoring sub-areas, determine the boundary lines between the H monitoring sub-areas according to the planning map, determine the initial boundary area based on the boundary lines and a preset standard distance, obtain all image data of the initial boundary area, and generate an image data set, wherein the standard distance is the distance in the vertical direction of the boundary line, and the image data set includes H initial boundary images.

[0108] The method of processing the image data set to determine the boundary image set includes:

[0109] Based on the edge detection algorithm, the boundary lines between each monitoring sub-area are identified, and the image data set is cropped according to the preset standard distance through the image segmentation algorithm to obtain H area cropped images. The H area cropped images are filtered based on the edge-preserving smoothing algorithm to obtain a boundary image set.

[0110] It should be noted that, in this embodiment, the image data set includes H initial boundary images, and the initial boundary images do not completely correspond to the initial boundary areas. Usually, in order to ensure the integrity of the image, the shooting range should be appropriately expanded during the process of obtaining the initial boundary images, so that each initial boundary image not only contains the entire content of the initial boundary area, but also can capture a part of the adjacent area. This approach helps to ensure that the boundary lines can be identified more accurately during the image segmentation and edge detection process, and effectively determine whether there are cross-regional objects.

[0111] Methods for determining whether there is a cross-region object based on a boundary image set include:

[0112] The boundary image set is input into the pre-built object detection model to obtain the detection results, which include cross-region and non-cross-region.

[0113] The object detection model construction method includes:

[0114] Obtain Q sets of training data, where Q is a positive integer greater than 1, and the training data includes a historical boundary image set and historical detection results;

[0115] The historical boundary image set and historical detection results are used as sample sets, the sample sets are divided into training sets and test sets, and a classifier is constructed;

[0116] The historical boundary image set in the training set is used as input data, and the historical detection results in the training set are used as output data to train the classifier to obtain an initial classifier;

[0117] The initial classifier is tested using the test set, and a classifier that meets the preset accuracy is output as the object detection model. The classifier is preferably a naive Bayes model or a support vector machine model.

[0118] Model building module: when there is a cross-region object, it is used to obtain a target image set corresponding to the cross-region object from the boundary image set, and to build a corresponding target 3D model according to the target image set;

[0119] In this embodiment, the target image set refers to the structural image set of a cross-regional object. An exemplary cross-regional object may be a transportation pipeline in a park. The structural image set may be multiple perspective images of the pipeline in each monitoring sub-region, images of the connection points or interface parts of the pipeline when crossing the sub-region boundary, and endoscopic images of the inside of the pipeline.

[0120] The method for obtaining a target image set corresponding to a cross-region object includes:

[0121] Identify cross-regional objects, obtain unique identification information corresponding to the cross-regional objects, obtain the corresponding standard image set based on the unique identification information, determine the corresponding shooting device information based on the standard image set, mobilize the corresponding shooting device based on the shooting device information, obtain the corresponding real-time image set and unify the format, and use the real-time image set in the unified format as the target image set.

[0122] The above-mentioned unique identification information can be an object ID number or a geometric feature fingerprint. It can be understood that the geometric feature fingerprint refers to the unique geometric shape, texture or surface feature of a cross-regional object. Taking the cross-regional object as a conveying pipeline as an example, the geometric feature fingerprint can be the outer diameter, shape, color, surface texture, etc. of the conveying pipeline. Identifying cross-regional objects means that feature recognition is performed on cross-regional objects in a boundary image set to obtain the corresponding geometric feature fingerprint. Identifying cross-regional objects can be achieved using existing computer vision algorithms. The computer vision algorithm can be a feature point detection algorithm (such as S IFT, SURF, ORB, etc.) to extract key feature points in the image. These feature points can describe the geometric shape, texture or surface features of the object.

[0123] The method for constructing a corresponding target three-dimensional model according to a target image set includes:

[0124] The target image set is identified to obtain the device model, a pre-built initial three-dimensional model is determined based on the device model, and a corresponding target three-dimensional model is constructed according to the target image set, simulation analysis software and the initial three-dimensional model, wherein the initial three-dimensional model at least includes a structural three-dimensional model.

[0125] The method of constructing the corresponding target three-dimensional model includes:

[0126] Structural feature data is obtained based on the target image set, and the structural feature data is input into the simulation analysis software. Then, the initial three-dimensional model is simulated and corrected according to the simulation analysis software to construct the corresponding target three-dimensional model.

[0127] A second processing module: used for generating a target frame block subset in response to a trigger operation acting on the interactive control, wherein the target frame block subset includes at least one target model frame block;

[0128] In this embodiment, the interactive control refers to a three-dimensional display control. The staff can use the interactive control to rotate the target three-dimensional model. Each side of the target three-dimensional model can be clearly displayed, thereby improving the visualization effect of the target three-dimensional model. The model frame block refers to a single-frame image data block sampled or generated at a certain time interval during the three-dimensional display process of the target three-dimensional model. These frame blocks represent the appearance and details of the three-dimensional model at a specific viewing angle and state. It can be understood that there are several model frame blocks on each side of the target three-dimensional model.

[0129] The method of generating a target frame block subset includes:

[0130] In response to a first trigger operation applied to the interactive control, a first model frame block set is output according to the first trigger operation, in response to a second trigger operation and obtaining a corresponding second model frame block set, a target frame block subset is generated based on the first model frame block set and the second model frame block set.

[0131] Interaction module: used to decode the target model frame block to obtain the shared model frame block, and display the target three-dimensional model in the interaction interface according to the shared model frame block.

[0132] In this embodiment, since the target three-dimensional models in this embodiment are all displayed by the interactive interface, but the interaction for fault monitoring requires real-time performance, the target three-dimensional models in this embodiment are all generated and displayed in real time, so the target three-dimensional models are all transmitted in real time, but due to reasons such as network speed and display memory, delays, freezes or inability to perform high-quality rendering are prone to occur during the display of the target three-dimensional model. Therefore, this embodiment first displays some model frame blocks, that is, displays the first model frame block set in the first perspective. When the staff switches to the second perspective, the target model frame blocks are first obtained, decoded and pre-stored. In this way, in the process of the next step of transmission, there is no need to transmit the target model frame blocks, and only the pre-stored target model frame blocks need to be displayed.

[0133] The target model frame block can be decoded by a corresponding software decoder, such as FFmpeg or DirectX Video Accession.

[0134] It should be added that, in the process of displaying the target three-dimensional model, this embodiment does not only rely on the shared model frame blocks for display, but also displays it in combination with the frame blocks corresponding to the model surface to be displayed.

[0135] Example 3

[0136] like Figure 5As shown, this embodiment discloses a computer-readable storage medium, on which a computer program is stored. When the computer program is executed, the above-mentioned intelligent interactive control method based on a three-dimensional engine is implemented.

[0137] The above formulas are all dimensionless and numerical calculations. The formula is a formula for the most recent real situation obtained by collecting a large amount of data and performing software simulation. The preset parameters, weights and thresholds in the formula are set by technicians in this field according to actual conditions.

[0138] The above embodiments can be implemented in whole or in part by software, hardware, firmware or any other combination. When implemented by software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website site, computer, server or data center to another website site, computer, server or data center through a wired network or a wireless network. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a tape), an optical medium (for example, a DVD) or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.

[0139] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed in the present invention can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0140] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0141] In the several embodiments provided by the present invention, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the units is only one, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0142] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0143] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0144] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

[0145] Finally: The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. An intelligent interactive control method based on a three-dimensional engine, applied to an interactive interface, displaying interactive controls through the interactive interface, characterized in that: include: Obtain image data sets of H monitoring sub-areas, process the image data sets to determine a boundary image set, and determine whether there is a cross-area object based on the boundary image set, where H is an integer greater than 1; When there is a cross-region object, a target image set corresponding to the cross-region object is obtained from the boundary image set, and a corresponding target three-dimensional model is constructed according to the target image set; In response to a trigger operation acting on the interactive control, a target frame block subset is generated, wherein the target frame block subset includes at least one target model frame block; The target model frame block is decoded to obtain a shared model frame block, and the target three-dimensional model is displayed in the interactive interface according to the shared model frame block.

2. The intelligent interactive control method based on a three-dimensional engine according to claim 1, characterized in that: The method for obtaining an image data set comprises: Obtain a planning map corresponding to H monitoring sub-areas, determine the boundary lines between the H monitoring sub-areas according to the planning map, determine the initial boundary area based on the boundary lines and a preset standard distance, obtain all image data of the initial boundary area, and generate an image data set, wherein the standard distance is the distance in the vertical direction of the boundary line, and the image data set includes H initial boundary images.

3. The intelligent interactive control method based on a three-dimensional engine according to claim 2, characterized in that: The method for processing an image data set to determine a boundary image set comprises: Based on the edge detection algorithm, the boundary lines between each monitoring sub-area are identified, and the image data set is cropped according to the preset standard distance through the image segmentation algorithm to obtain H area cropped images. The H area cropped images are filtered based on the edge-preserving smoothing algorithm to obtain a boundary image set.

4. The intelligent interactive control method based on a three-dimensional engine according to claim 3, characterized in that: The method for determining whether there is a cross-region object based on a boundary image set includes: Input the boundary image set into the pre-built object detection model to obtain the detection results, including cross-region and non-cross-region results; The object detection model construction method includes: Obtain Q sets of training data, where Q is a positive integer greater than 1, and the training data includes a historical boundary image set and historical detection results; The historical boundary image set and historical detection results are used as sample sets, the sample sets are divided into training sets and test sets, and a classifier is constructed; The historical boundary image set in the training set is used as input data, and the historical detection results in the training set are used as output data to train the classifier to obtain an initial classifier; The initial classifier is tested using the test set, and the classifier that meets the preset accuracy is output as the object detection model.

5. The intelligent interactive control method based on a three-dimensional engine according to claim 1, characterized in that: The method for obtaining a target image set corresponding to a cross-region object includes: Identify cross-regional objects, obtain unique identification information corresponding to the cross-regional objects, obtain the corresponding standard image set based on the unique identification information, determine the corresponding shooting device information based on the standard image set, mobilize the corresponding shooting device based on the shooting device information, obtain the corresponding real-time image set and unify the format, and use the real-time image set in the unified format as the target image set.

6. The intelligent interactive control method based on a three-dimensional engine according to claim 5, characterized in that: The method for constructing a corresponding target three-dimensional model according to a target image set comprises: The target image set is identified to obtain the device model, a pre-built initial three-dimensional model is determined based on the device model, and a corresponding target three-dimensional model is constructed according to the target image set, simulation analysis software and the initial three-dimensional model, wherein the initial three-dimensional model at least includes a structural three-dimensional model.

7. The intelligent interactive control method based on a three-dimensional engine according to claim 6, characterized in that: The method for constructing the corresponding target three-dimensional model comprises: Structural feature data is obtained based on the target image set, and the structural feature data is input into the simulation analysis software. Then, the initial three-dimensional model is simulated and corrected according to the simulation analysis software to construct the corresponding target three-dimensional model.

8. The intelligent interactive control method based on a three-dimensional engine according to claim 1, characterized in that: The method for generating a target frame block subset comprises: In response to a first trigger operation applied to the interactive control, a first model frame block set is output according to the first trigger operation, in response to a second trigger operation and obtaining a corresponding second model frame block set, a target frame block subset is generated based on the first model frame block set and the second model frame block set.

9. An intelligent interactive control system based on a three-dimensional engine, which is used to implement an intelligent interactive control method based on a three-dimensional engine as claimed in any one of claims 1 to 8, characterized in that: include: The first processing module is used to obtain image data sets of H monitoring sub-areas, process the image data sets to determine boundary image sets, and determine whether there is a cross-area object based on the boundary image sets, where H is an integer greater than 1; Model building module: when there is a cross-region object, it is used to obtain a target image set corresponding to the cross-region object from the boundary image set, and to build a corresponding target 3D model according to the target image set; A second processing module: used for generating a target frame block subset in response to a trigger operation acting on the interactive control, wherein the target frame block subset includes at least one target model frame block; Interaction module: used to decode the target model frame block to obtain the shared model frame block, and display the target three-dimensional model in the interaction interface according to the shared model frame block.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed, the intelligent interactive control method based on a three-dimensional engine as described in any one of claims 1 to 8 is implemented.

Images