Model display method, device, electronic device and computer-readable medium

Abstract

Embodiments of the present disclosure disclose a model display method, apparatus, electronic device, and computer-readable medium. A specific implementation of the method includes: obtaining a first three-dimensional model corresponding to a two-dimensional drawing; collecting the two-dimensional drawing corresponding to the interactive screen according to a camera group included in the interactive screen to obtain a two-dimensional drawing image; identifying a target drawing identifier included in the two-dimensional drawing image to obtain a drawing identifier recognition result; determining position information corresponding to the target drawing identifier according to the target drawing identifier characterized by the drawing identifier recognition result; extracting a second three-dimensional model corresponding to the position information from the first three-dimensional model according to the position information and the interactive screen information of the interactive screen; and displaying the second three-dimensional model on the interactive screen. This implementation improves the interactivity and timeliness of the displayed model, and further enhances the data transparency, response timeliness, and integrated management on the factory management side by displaying different real-time data.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the field of computer technologies, and more particularly, to a method and apparatus for model display, an electronic device, and a computer-readable medium. Background Art

[0002] When displaying a model, there is an increasing demand for the interactivity of the model and the timeliness of model response. Currently, when displaying a model, one way adopted is to display the model through a two-dimensional drawing and annotate the model information in the two-dimensional drawing.

[0003] However, when the above method is used for model display, the following technical problems often exist:

[0004] When a model is displayed through a two-dimensional drawing, the interactivity of the displayed model is low, and it cannot respond to user operations, resulting in a poor user experience.

[0005] The above information disclosed in this background art section is only used to enhance the understanding of the background of the inventive concept, and thus, it may include information that does not form the prior art known to those of ordinary skill in the art in this country. Summary of the Invention

[0006] This summary of the disclosure is intended to introduce concepts in a brief form, which will be described in detail in the following detailed implementation section. This summary of the disclosure is not intended to identify key features or essential features of the claimed technical solution, nor is it intended to be used to limit the scope of the claimed technical solution.

[0007] Some embodiments of the present disclosure provide a method and apparatus for model display, an electronic device, and a computer-readable medium to solve one or more of the technical problems mentioned in the above background art section.

[0008] In a first aspect, some embodiments of the present disclosure provide a model display method, the method comprising: in response to detecting a startup operation on an interactive screen included in the above-mentioned model interaction device, acquiring a first three-dimensional model corresponding to the above-mentioned two-dimensional drawing; collecting a two-dimensional drawing corresponding to the interactive screen according to a camera group included in the interactive screen to obtain a two-dimensional drawing image; identifying a target drawing identifier included in the above-mentioned two-dimensional drawing image to obtain a drawing identifier recognition result; determining position information corresponding to the above-mentioned target drawing identifier according to the target drawing identifier characterized by the above-mentioned drawing identifier recognition result; intercepting a second three-dimensional model corresponding to the above-mentioned position information from the above-mentioned first three-dimensional model according to the above-mentioned position information and the interactive screen information of the interactive screen; displaying the above-mentioned second three-dimensional model on the above-mentioned interactive screen; in response to detecting a selection operation by a user on any three-dimensional module in the second three-dimensional model displayed on the above-mentioned interactive screen, generating a three-dimensional module information display page, where the above-mentioned three-dimensional module information display page is a page for displaying three-dimensional module information of the selected three-dimensional module; and displaying the above-mentioned three-dimensional module information display page on the above-mentioned interactive screen.

[0009] In a second aspect, some embodiments of the present disclosure provide a model display device, the device comprising: an acquisition unit configured to acquire a first three-dimensional model corresponding to the above-mentioned two-dimensional drawing in response to detecting a startup operation on an interactive screen included in the above-mentioned model interaction device; a collection unit configured to collect a two-dimensional drawing corresponding to the interactive screen according to a camera group included in the interactive screen to obtain a two-dimensional drawing image; an identification unit configured to identify a target drawing identifier included in the above-mentioned two-dimensional drawing image to obtain a drawing identifier recognition result; a determination unit configured to determine position information corresponding to the above-mentioned target drawing identifier according to the target drawing identifier characterized by the above-mentioned drawing identifier recognition result; an interception unit configured to intercept a second three-dimensional model corresponding to the above-mentioned position information from the above-mentioned first three-dimensional model according to the above-mentioned position information and the interactive screen information of the interactive screen; a first display unit configured to display the above-mentioned second three-dimensional model on the above-mentioned interactive screen; a generation unit configured to generate a three-dimensional module information display page in response to detecting a selection operation by a user on any three-dimensional module in the second three-dimensional model displayed on the above-mentioned interactive screen, where the above-mentioned three-dimensional module information display page is a page for displaying three-dimensional module information of the selected three-dimensional module;

[0010] A second display unit configured to display the above-mentioned three-dimensional module information display page on the above-mentioned interactive screen.

[0011] In a third aspect, some embodiments of the present disclosure provide an electronic device, including: one or more processors; a storage device storing one or more programs thereon, and when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation manner of the first aspect above.

[0012] In a fourth aspect, some embodiments of the present disclosure provide a computer-readable medium storing a computer program thereon, wherein when the program is executed by a processor, the method described in any implementation manner of the first aspect above is implemented.

[0013] The above various embodiments of the present disclosure have the following beneficial effects: Through the model display method of some embodiments of the present disclosure, the interactivity of the model and the timeliness of the model response are enhanced, thereby improving the user experience. Specifically, the reasons for the low interactivity of the model and the poor user experience are as follows: When the model is displayed through two-dimensional drawings, the interactivity of the displayed model is low, and it cannot respond to user operations, resulting in a poor user experience. Based on this, the model display method of some embodiments of the present disclosure first, in response to detecting a startup operation for the interactive screen included in the above model interaction device, obtains the first three-dimensional model corresponding to the above two-dimensional drawing. Thus, the three-dimensional model corresponding to the entire two-dimensional drawing can be obtained. Secondly, according to the camera group included in the interactive screen, the two-dimensional drawing corresponding to the interactive screen is collected to obtain a two-dimensional drawing image. Thus, the part of the drawing that the user wants to view can be determined. Then, the target drawing identifier included in the two-dimensional drawing image is recognized to obtain a drawing identifier recognition result; according to the target drawing identifier represented by the drawing identifier recognition result, the position information corresponding to the target drawing identifier is determined. Thus, the position of the part viewed by the user in the three-dimensional model can be determined. Then, according to the position information and the interactive screen information of the interactive screen, the second three-dimensional model corresponding to the position information is intercepted from the first three-dimensional model. Thus, the part of the three-dimensional model that the user needs to view can be split and intercepted. After that, the second three-dimensional model is displayed on the interactive screen. Thus, the part of the three-dimensional model corresponding to the two-dimensional drawing can be displayed. Finally, in response to detecting a selection operation on any three-dimensional module in the second three-dimensional model displayed on the interactive screen by the user, a three-dimensional module information display page is generated; the three-dimensional module information display page is displayed on the interactive screen. Thus, by interacting with different device modules in the three-dimensional model, the module information corresponding to each device module can be displayed, improving the interactivity and timeliness of the displayed model, and further, by displaying different real-time data, the data transparency, response timeliness, and management integration of the factory management side can be improved. Description of the Drawings

[0014] In combination with the accompanying drawings and with reference to the following specific embodiments, the above and other features, advantages and aspects of the various embodiments of the present disclosure will become more apparent. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic and the elements and elements are not necessarily drawn to scale.

[0015] Figure 1 is a flowchart of some embodiments of a model display method according to the present disclosure;

[0016] Figure 2 is a schematic structural diagram of some embodiments of a model display device according to the present disclosure;

[0017] Figure 3 is a schematic structural diagram of an electronic device suitable for implementing some embodiments of the present disclosure;

[0018] Figure 4 is a physical diagram of a model interaction device of some embodiments of a model display method according to the present disclosure. Specific Embodiments

[0019] The embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although some embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. On the contrary, these embodiments are provided to more thoroughly and completely understand the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for illustrative purposes and are not used to limit the protection scope of the present disclosure.

[0020] In addition, it should be noted that for the sake of convenience of description, only the parts related to the relevant invention are shown in the drawings. Without conflict, the embodiments in the present disclosure and the features in the embodiments can be combined with each other.

[0021] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or mutual dependence relationship of the functions performed by these devices, modules or units.

[0022] It should be noted that the modifications of "one" and "multiple" mentioned in the present disclosure are illustrative rather than restrictive. Those skilled in the art should understand that unless otherwise clearly specified in the context, it should be understood as "one or more".

[0023] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only for illustrative purposes and are not used to limit the scope of these messages or information.

[0024] The present disclosure will be described in detail below with reference to the drawings and in combination with the embodiments.

[0025] Figure 1 Flow 100 of some embodiments of a model display method according to the present disclosure is shown. The model display method includes the following steps:

[0026] Step 101, in response to detecting a startup operation for an interactive screen included in a model interaction device, obtain a first 3D model corresponding to a 2D drawing.

[0027] In some embodiments, an execution subject (such as a server) of the model display method may, in response to detecting a startup operation for the interactive screen included in the above-mentioned model interaction device, obtain a first 3D model corresponding to the above-mentioned 2D drawing. In practice, the above-mentioned execution subject may obtain the first 3D model corresponding to the above-mentioned 2D drawing from a target database. Among them, the above-mentioned model interaction device includes an interactive screen and a 2D drawing. The above-mentioned 2D drawing may be a drawing for displaying a device scene. The above-mentioned 2D drawing can be replaced. One side of the above-mentioned interactive screen is a touchable display screen, and the other side includes a camera group for photographing the 2D drawing. The above-mentioned model interaction device further includes a sliding track, and the above-mentioned interactive screen can slide along the track direction through the above-mentioned sliding track. The above-mentioned target database may be a database for storing 3D models. The above-mentioned first 3D model may be a 3D model corresponding to the above-mentioned pre-generated 2D drawing. Some embodiments of the model interaction device of the model display method of the present disclosure are as Figure 4 shown.

[0028] Step 102, collect the 2D drawing corresponding to the interactive screen according to the camera group included in the interactive screen to obtain a 2D drawing image.

[0029] In some embodiments, the above-mentioned execution subject may collect the 2D drawing corresponding to the above-mentioned interactive screen according to the camera group included in the above-mentioned interactive screen to obtain a 2D drawing image.

[0030] Optionally, after step 102, the following steps are further included:

[0031] The first step is to perform image segmentation processing on the 2D drawing image to generate a set of segmented drawing images.

[0032] In some embodiments, the above-mentioned execution entity may perform image segmentation processing on the above-mentioned two-dimensional drawing image to generate a set of segmented drawing images. In practice, first, the above-mentioned execution entity may perform contour detection processing on the above-mentioned two-dimensional drawing image to display the contours of each building and fixed facility in the two-dimensional drawing image. Secondly, the above-mentioned two-dimensional drawing image is subjected to image segmentation processing according to the displayed contours and the edges of the above-mentioned two-dimensional drawing image. Here, the above-mentioned contour detection processing may be to detect the contours of the buildings and fixed facilities displayed in the two-dimensional drawing image through a contour detection algorithm based on OpenCV.

[0033] In the second step, obtain a preset three-dimensional module set from the target database.

[0034] In some embodiments, the above-mentioned execution entity may obtain a preset three-dimensional module set from the target database, where the preset three-dimensional modules in the above-mentioned preset three-dimensional module set correspond to module numbers and module sizes. The preset three-dimensional modules in the above-mentioned preset three-dimensional module set may be modules of three-dimensional models of pre-generated buildings and fixed facilities.

[0035] In the third step, for each segmented drawing image in the set of segmented drawing images, perform the following selection steps:

[0036] In the first selection step, select a preset three-dimensional module that meets the first preset condition from the preset three-dimensional module set as the target three-dimensional module.

[0037] In some embodiments, the above-mentioned execution entity may select a preset three-dimensional module that meets the first preset condition from the above-mentioned preset three-dimensional module set as the target three-dimensional module.

[0038] In practice, it is found that when the model is displayed through the interactive screen, the three-dimensional modules at the edge of the interactive screen cannot be fully displayed, and some three-dimensional modules are not stored, resulting in an incomplete three-dimensional model displayed, and thus the interactivity and response timeliness of the three-dimensional model are relatively low.

[0039] In practice, the above-mentioned execution entity may select a preset three-dimensional module that meets the first preset condition from the above-mentioned preset three-dimensional module set through the following sub-steps:

[0040] In the first sub-step, obtain a preset two-dimensional module image set. Among them, the preset two-dimensional module images in the above-mentioned preset two-dimensional module image set correspond to module numbers. The preset two-dimensional module images in the above-mentioned preset two-dimensional module image set may be floor plans of pre-set buildings and fixed facilities.

[0041] The second sub-step is to input each of the above-mentioned preset two-dimensional module images in the preset two-dimensional module image set and the above-mentioned segmented drawing image into a pre-trained similarity model to obtain the drawing similarity. Among them, the above-mentioned similarity model can be the cosine similarity formula.

[0042] The third sub-step is to select the highest drawing similarity from the obtained various drawing similarities as the target similarity.

[0043] The fourth sub-step is to select a preset three-dimensional module corresponding to the above-mentioned target similarity from the above-mentioned preset three-dimensional module set as the target three-dimensional module. In practice, a preset three-dimensional module with the same module number as the module number of the preset two-dimensional module image corresponding to the above-mentioned target similarity can be selected as the target three-dimensional module.

[0044] Optionally, before the first sub-step, the following steps are further included:

[0045] The first step is to perform edge line detection processing on the above-mentioned segmented drawing image to generate a detection result.

[0046] In some embodiments, the above-mentioned execution subject can perform edge line detection processing on the above-mentioned segmented drawing image to generate a detection result. Among them, the above-mentioned edge line detection can be to detect the edge lines of the building or fixed facilities shown in the above-mentioned segmented drawing image.

[0047] The second step is to generate a predicted three-dimensional module according to the above-mentioned preset two-dimensional module image set in response to the detection result indicating that the edge line corresponding to the above-mentioned segmented drawing image does not meet the edge line closing condition.

[0048] In some embodiments, the above-mentioned execution subject can generate a predicted three-dimensional module according to the above-mentioned preset two-dimensional module image set in response to the detection result indicating that the edge line corresponding to the above-mentioned segmented drawing image does not meet the edge line closing condition. Among them, the above-mentioned edge line closing condition can be that the edge line of the building or fixed facilities shown in the above-mentioned segmented drawing image is not in a closed state (the building or fixed facilities shown in the segmented drawing image are incomplete).

[0049] The third step is to perform cutting processing on the above-mentioned predicted three-dimensional module according to the above-mentioned segmented drawing image to generate a cut three-dimensional module.

[0050] In some embodiments, the above-mentioned execution subject can perform cutting processing on the above-mentioned predicted three-dimensional module according to the above-mentioned segmented drawing image to generate a cut three-dimensional module. In practice, the above-mentioned cutting processing can be to cut the above-mentioned predicted three-dimensional module according to the part shown in the above-mentioned segmented drawing image.

[0051] In the fourth step, determine the above-mentioned three-dimensional module after cutting as the adjusted three-dimensional module.

[0052] In some embodiments, the above-mentioned execution subject may determine the above-mentioned three-dimensional module after cutting as the adjusted three-dimensional module.

[0053] The above first sub-step - the fourth step, as an inventive point of the embodiment of the present disclosure, solves the technical problem that "when a model is displayed through an interactive screen, the three-dimensional modules at the edge of the interactive screen cannot be fully displayed, and some three-dimensional modules are not stored, resulting in an incomplete three-dimensional model being displayed, and further resulting in low interactivity and corresponding timeliness of the three-dimensional model". The reasons for the poor interactivity and response timeliness of the three-dimensional model are as follows: when a model is displayed through an interactive screen, the three-dimensional modules at the edge of the interactive screen cannot be fully displayed, and some three-dimensional modules are not stored, resulting in an incomplete three-dimensional model being displayed, and further resulting in low interactivity and corresponding timeliness of the three-dimensional model. If the above factors are solved, the effect of improving the interactivity and response timeliness of the three-dimensional model can be achieved. To achieve this effect, the present disclosure first obtains a preset two-dimensional module image set. Thus, two-dimensional module images of each possible two-dimensional module set in advance can be obtained. Second, for each preset two-dimensional module image in the above preset two-dimensional module image set, input the above preset two-dimensional module image and the above segmented drawing image into a pre-trained similarity model to obtain a drawing similarity; select the highest drawing similarity from the obtained drawing similarities as the target similarity; select the preset three-dimensional module corresponding to the above target similarity from the above preset three-dimensional module set as the target three-dimensional module. Thus, the three-dimensional module corresponding to the segmented drawing image can be determined through the similarity of the images of the preset two-dimensional modules. Third, perform edge line detection processing on the above segmented drawing image to generate a detection result. Thus, it can be determined whether the segmented drawing image is complete. Fourth, in response to the detection result indicating that the edge line corresponding to the above segmented drawing image does not meet the edge line closing condition, generate a predicted three-dimensional module according to the above preset two-dimensional module image set. Thus, the three-dimensional module corresponding to the incomplete drawing image can be predicted. Fifth, perform cutting processing on the above predicted three-dimensional module according to the above segmented drawing image to generate a three-dimensional module after cutting; determine the above three-dimensional module after cutting as the adjusted three-dimensional module. Thus, by cutting the three-dimensional module and then displaying the cut three-dimensional module on the interactive screen, it is possible to avoid an incomplete three-dimensional model being displayed, and further improve the interactivity and response timeliness of the three-dimensional model.

[0054] Second selection step, adjust the proportion and direction of the target three-dimensional module according to the position of the segmented drawing image in the two-dimensional drawing image to obtain the adjusted three-dimensional module.

[0055] In some embodiments, the above-mentioned execution entity may adjust the scale and orientation of the above-mentioned target three-dimensional module according to the position of the above-mentioned segmented drawing image in the above-mentioned two-dimensional drawing image to obtain an adjusted three-dimensional module. In practice, the size and orientation of the target three-dimensional module may be adjusted according to the distance between the above-mentioned segmented drawing image and the edge of the above-mentioned two-dimensional drawing image.

[0056] Step 4: Based on the obtained adjusted three-dimensional modules, perform model reconstruction on the two-dimensional drawing image to generate a reconstructed three-dimensional model, and determine the reconstructed three-dimensional model as the second three-dimensional model.

[0057] In some embodiments, the above-mentioned execution entity may perform model reconstruction on the above-mentioned two-dimensional drawing image based on the obtained adjusted three-dimensional modules to generate a reconstructed three-dimensional model, and determine the reconstructed three-dimensional model as the second three-dimensional model.

[0058] Step 103: Identify the target drawing identifier included in the two-dimensional drawing image to obtain a drawing identifier recognition result.

[0059] In some embodiments, the above-mentioned execution entity may identify the target drawing identifier included in the above-mentioned two-dimensional drawing image to obtain a drawing identifier recognition result. Here, the pre-set position representation displayed in the above-mentioned two-dimensional drawing image may be identified.

[0060] In practice, the following steps may be used to identify the target drawing identifier included in the two-dimensional drawing image to obtain a drawing identifier recognition result:

[0061] Step 1: Crop each identifier displayed in the above-mentioned two-dimensional drawing image to obtain a cropped identifier group. Here, the contour detection algorithm based on OpenCV may be used to determine each identifier displayed in the two-dimensional drawing image.

[0062] Step 2: Input each cropped identifier in the above-mentioned cropped identifier group into a pre-trained identifier recognition model to obtain a drawing identifier recognition result.

[0063] Optionally, the above-mentioned identifier recognition model may be trained through the following steps:

[0064] Step 1: Obtain a sample set.

[0065] In some embodiments, the above-mentioned execution entity may obtain a sample set. Among them, the samples in the above-mentioned sample set include sample cropped identifiers and sample drawing identifier recognition results corresponding to the above-mentioned sample cropped identifiers.

[0066] Step 2: Select samples from the above-mentioned sample set.

[0067] In some embodiments, the above-mentioned execution entity may select samples from the above-mentioned sample set. Here, the above-mentioned execution entity may randomly select samples from the above-mentioned sample set.

[0068] In the third step, input the above-mentioned samples into the initial network model to obtain the drawing identification result corresponding to the above-mentioned samples.

[0069] In some embodiments, the above-mentioned execution entity may input the above-mentioned samples into the initial network model to obtain the drawing identification result corresponding to the above-mentioned samples. Among them, the above-mentioned initial neural network may be a classification model capable of obtaining the drawing identification result according to the cropping identification.

[0070] In the fourth step, determine the loss value between the above-mentioned drawing identification result and the sample drawing identification result included in the above-mentioned samples.

[0071] In some embodiments, the above-mentioned execution entity may determine the loss value between the above-mentioned drawing identification result and the sample drawing identification result included in the above-mentioned samples. In practice, based on a preset loss function, determine the loss value between the above-mentioned drawing identification result and the sample drawing identification result included in the above-mentioned samples. For example, the above-mentioned preset loss function may be a cross-entropy loss function.

[0072] In the fifth step, in response to the above-mentioned loss value being greater than or equal to a preset threshold, adjust the network parameters of the above-mentioned initial network model.

[0073] In some embodiments, the above-mentioned execution entity may, in response to the above-mentioned loss value being greater than or equal to a preset threshold, adjust the network parameters of the above-mentioned initial network model. Here, there is no limitation on the setting of the preset threshold. For example, the difference between the loss value and the preset threshold may be calculated to obtain the loss difference. On this basis, methods such as backpropagation and stochastic gradient descent are used to forward the error value from the last layer of the model to adjust the parameters of each layer. Of course, according to needs, the method of network freezing (dropout) may also be adopted to keep the network parameters of some layers unchanged and not adjust them. There is no limitation on this.

[0074] Optionally, in response to the above-mentioned loss value being less than the above-mentioned preset threshold, determine the initial network model as the identification model.

[0075] In some embodiments, the above-mentioned execution entity may, in response to the above-mentioned loss value being less than the above-mentioned preset threshold, determine the initial network model as the identification model.

[0076] Step 104, determine the position information corresponding to the target drawing identification according to the target drawing identification characterized by the drawing identification result.

[0077] In some embodiments, the above-mentioned execution entity may determine the position information corresponding to the target drawing identifier based on the target drawing identifier characterized by the above-mentioned drawing identifier recognition result. In practice, the above-mentioned execution entity may determine the position of the above-mentioned target drawing identifier in the above-mentioned 3D model.

[0078] In practice, the position information corresponding to the target drawing identifier may be determined through the following steps:

[0079] First step, obtain the interactive screen information and the interactive screen display image of the above-mentioned interactive screen. Among them, the above-mentioned interactive screen information may include the length and width of the interactive screen. The above-mentioned interactive screen display image may be a blank image representing the display range of the interactive screen. For example, if the interactive screen is rectangular, the above-mentioned interactive screen display image may be a blank rectangular image.

[0080] Second step, generate an interactive screen coordinate system based on the above-mentioned interactive screen display image. In practice, the lower left corner vertex of the above-mentioned interactive screen display image may be used as the origin of the coordinate system to establish a rectangular coordinate system.

[0081] Third step, determine the position information corresponding to the target drawing identifier according to the above-mentioned interactive screen coordinate system, the above-mentioned interactive screen length, and the above-mentioned interactive screen width. In practice, the position coordinates of the above-mentioned target drawing identifier in the above-mentioned interactive screen coordinate system may be determined as the position information.

[0082] Step 105, extract a second 3D model corresponding to the position information from the first 3D model according to the position information and the interactive screen information of the interactive screen.

[0083] In some embodiments, the above-mentioned execution entity may extract a second 3D model corresponding to the above-mentioned position information from the above-mentioned first 3D model according to the above-mentioned position information and the interactive screen information of the interactive screen. In practice, first, determine the first screen width and the second screen width according to the width of the interactive screen included in the above-mentioned interactive screen information. Among them, the above-mentioned first screen width may be the distance between one side of the above-mentioned interactive screen and the position represented by the above-mentioned position information. The above-mentioned second screen width may be the distance between the other side of the above-mentioned interactive screen and the position represented by the above-mentioned position information. Second, extract a partial 3D model corresponding to the above-mentioned first screen width and the second screen width from the above-mentioned first 3D model as the second 3D model.

[0084] Step 106, display the second 3D model on the interactive screen.

[0085] In some embodiments, the above-mentioned execution entity may display the above-mentioned second 3D model on the above-mentioned interactive screen.

[0086] Step 107: Generate a 3D module information display page in response to detecting a selection operation by the user on any 3D module in the second 3D model displayed on the interactive screen.

[0087] In some embodiments, the above-mentioned execution entity may generate a 3D module information display page in response to detecting a selection operation by the user on any 3D module in the second 3D model displayed on the above-mentioned interactive screen. Wherein, the above-mentioned 3D module information display page is a page for displaying the 3D module information of the selected 3D module.

[0088] Step 108: Display the 3D module information display page on the interactive screen.

[0089] In some embodiments, the above-mentioned execution entity may display the above-mentioned 3D module information display page on the above-mentioned interactive screen.

[0090] In practice, when moving the interactive screen to display the 3D model, obtaining the 2D drawing in real time through the camera group to generate the 3D model has relatively high requirements for the terminal and the network. In a poor network environment, the 3D model cannot be displayed in time, resulting in poor interactivity and response timeliness of the 3D model.

[0091] Optionally, after step 108, the following steps are further included:

[0092] First step: Detect in real time the movement operation on the interactive screen included in the above-mentioned model interaction device. In practice, the infrared sensor set in advance may be used to detect in real time whether the above-mentioned interactive screen moves.

[0093] Second step: In response to detecting the movement operation on the above-mentioned interactive screen and the movement distance of the above-mentioned interactive screen being equal to the preset movement distance, generate an updated second 3D model for display according to the camera group included in the above-mentioned interactive screen. Wherein, the above-mentioned preset movement distance may be the distance of the movement of the interactive screen set in advance.

[0094] Third step: Perform segmentation processing on the 2D drawing image corresponding to the above-mentioned updated second 3D model to generate a group of segmented 2D drawing images. In practice, first, the edge detection algorithm based on OpenCV may be used to perform edge detection on each device displayed in the 2D drawing image corresponding to the above-mentioned updated second 3D model. Secondly, image segmentation is performed according to the generated edges of each device to obtain a group of segmented 2D drawing images.

[0095] Fourth step: For each segmented 2D drawing image in the above-mentioned group of segmented 2D drawing images, perform the following processing steps:

[0096] The first processing step is to determine the similarity between the above-mentioned segmented two-dimensional drawing image and each preset two-dimensional module image in the above-mentioned preset two-dimensional module image set, and obtain a similarity set.

[0097] The second processing step is to select the preset two-dimensional module image with the highest corresponding similarity from the above-mentioned preset two-dimensional module image group as the target image.

[0098] The third processing step is to update the above-mentioned segmented two-dimensional drawing image according to the above-mentioned preset moving distance and the above-mentioned preset two-dimensional module image to obtain an updated image. In practice, the segmented two-dimensional drawing image generated after moving the device represented by the above-mentioned segmented two-dimensional drawing image according to the preset moving distance and the direction corresponding to the above-mentioned moving operation can be used as the updated image.

[0099] The fifth step is to perform a splicing process on the generated updated images to generate a spliced two-dimensional drawing image. In practice, the splicing process can be performed according to the positions of the segmented two-dimensional drawing images corresponding to each updated image in the above-mentioned two-dimensional drawing image.

[0100] The sixth step is to perform a preloading process according to the above-mentioned spliced two-dimensional drawing image to obtain a preloaded three-dimensional model. In practice, the above-mentioned preloading process can be to generate a three-dimensional model corresponding to the above-mentioned spliced two-dimensional drawing image and store it in the cache.

[0101] The seventh step is to, in response to detecting a moving operation on the above-mentioned interactive screen again, and the moving direction and moving distance of the above-mentioned interactive screen being the same as those of the previous moving operation, load the preloaded three-dimensional model stored in the cache for display.

[0102] The eighth step is to, in response to detecting a moving operation on the above-mentioned interactive screen again, and the moving direction of the above-mentioned interactive screen being opposite to that of the previous moving operation, redisplay the above-mentioned second three-dimensional model.

[0103] The above first step to eighth step, as an inventive point of the embodiment of the present disclosure, solves the technical problem that "when moving the interactive screen to display a 3D model, obtaining 2D drawings in real time through a camera group to generate a 3D model requires high requirements for the terminal and the network. In a poor network environment, the 3D model cannot be displayed in time, resulting in poor interactivity and response timeliness of the 3D model." The reasons for the poor interactivity and response timeliness of the 3D model are as follows: when moving the interactive screen to display a 3D model, obtaining 2D drawings in real time through a camera group to generate a 3D model requires high requirements for the terminal and the network. In a poor network environment, the 3D model cannot be displayed in time, resulting in poor interactivity and response timeliness of the 3D model. If the above factors are solved, the effect of improving the interactivity and response timeliness of the 3D model can be achieved. To achieve this effect, the present disclosure first detects a movement operation on the interactive screen included in the above model interaction device in real time. Thus, the movement of the interactive screen can be detected. Second, in response to detecting a movement operation on the above interactive screen and the movement distance of the above interactive screen being equal to a preset movement distance, an updated second 3D model is generated according to the camera group included in the above interactive screen for display. Thus, a 2D drawing image can be captured by the camera group included in the interactive screen, and the 3D model after the first movement can be displayed. Third, the 2D drawing image corresponding to the above updated second 3D model is segmented to generate a segmented 2D drawing image group. Thus, the drawings corresponding to each device in the 2D drawing image can be distinguished. Fourth, for each segmented 2D drawing image in the above segmented 2D drawing image group, the following processing steps are performed: determining the similarity between the above segmented 2D drawing image and each preset 2D module image in the above preset 2D module image set to obtain a similarity set; selecting the preset 2D module image with the highest corresponding similarity from the above preset 2D module image group as the target image; updating the above segmented 2D drawing image according to the above preset movement distance and the above preset 2D module image to obtain an updated image. Thus, the image of each device after moving a preset movement distance can be predicted. Fifth, the generated updated images are spliced to generate a spliced 2D drawing image. Thus, a drawing image that moves a preset movement distance in the overall moving direction can be generated. Sixth, preloading processing is performed according to the above spliced 2D drawing image to obtain a preloaded 3D model. Thus, the drawing image can be preloaded with a 3D model and stored. Also, because the 3D model is preloaded, the real-time generation of the 3D model during movement is avoided, thereby avoiding the situation where the 3D model cannot be displayed due to a poor network environment, and improving the interactivity and response timeliness of the 3D model.Seventh, in response to detecting a movement operation on the above interactive screen again, and the movement direction and distance of the above interactive screen being the same as the previous movement operation, load the pre-loaded 3D model stored in the cache for display; in response to detecting a movement operation on the above interactive screen again, and the movement direction of the above interactive screen being opposite to the previous movement operation, re-display the above second 3D model. Thereby, the interactivity of the 3D model and the timeliness of the response are improved. Furthermore, by displaying different real-time data, the data transparency, response timeliness, and integrated management on the factory management side can be enhanced.

[0104] The above various embodiments of the present disclosure have the following beneficial effects: Through the model display method of some embodiments of the present disclosure, the interactivity of the model and the timeliness of the model response are enhanced, thereby improving the user experience. Specifically, the reasons for the low interactivity of the model and the poor user experience are as follows: When the model is displayed through 2D drawings, the interactivity of the displayed model is low, and it cannot respond to user operations, resulting in a poor user experience. Based on this, the model display method of some embodiments of the present disclosure first, in response to detecting a startup operation on the interactive screen included in the above model interaction device, obtains the first 3D model corresponding to the above 2D drawing. Thereby, the 3D model corresponding to the entire 2D drawing can be obtained. Secondly, according to the camera group included in the above interactive screen, collect the 2D drawing corresponding to the above interactive screen to obtain a 2D drawing image. Thereby, the part of the drawing that the user wants to view can be determined. Then, identify the target drawing identifier included in the above 2D drawing image to obtain a drawing identifier recognition result; determine the position information corresponding to the above target drawing identifier according to the target drawing identifier represented by the above drawing identifier recognition result. Thereby, the position of the part viewed by the user in the 3D model can be determined. Then, according to the above position information and the interactive screen information of the above interactive screen, intercept the second 3D model corresponding to the above position information from the above first 3D model. Thereby, the part of the 3D model that the user needs to view can be split and intercepted. After that, display the above second 3D model on the above interactive screen. Thereby, the part of the 3D model corresponding to the 2D drawing can be displayed. Finally, in response to detecting a selection operation on any 3D module in the second 3D model displayed on the above interactive screen by the user, generate a 3D module information display page; display the above 3D module information display page on the above interactive screen. Thereby, interactivity can be achieved by selecting different device modules in the 3D model, and the module information corresponding to each device module can be displayed, improving the interactivity and timeliness of the displayed model. Furthermore, by displaying different real-time data, the data transparency, response timeliness, and integrated management on the factory management side can be enhanced.

[0105] For further reference Figure 2, as an implementation of the methods shown in the above figures, the present disclosure provides some embodiments of a model display device, and these device embodiments correspond to Figure 1 the method embodiments shown, and the model display device can be specifically applied to various electronic devices.

[0106] As Figure 2 shown, the model display device 200 of some embodiments includes: an acquisition unit 201, a collection unit 202, an identification unit 203, a determination unit 204, a truncation unit 205, a first display unit 206, a generation unit 207, and a second display unit 208. Among them, the acquisition unit 201 is configured to, in response to detecting a startup operation for the interactive screen included in the above model interaction device, acquire a first three-dimensional model corresponding to the above two-dimensional drawing; the collection unit 202 is configured to collect the two-dimensional drawing corresponding to the above interactive screen according to the camera group included in the above interactive screen to obtain a two-dimensional drawing image; the identification unit 203 is configured to identify the target drawing identifier included in the above two-dimensional drawing image to obtain a drawing identifier recognition result; the determination unit 204 is configured to determine the position information corresponding to the above target drawing identifier according to the target drawing identifier characterized by the above drawing identifier recognition result; the truncation unit 205 is configured to truncate a second three-dimensional model corresponding to the above position information from the above first three-dimensional model according to the above position information and the interactive screen information of the above interactive screen; the display unit 206 is configured to display the above second three-dimensional model on the above interactive screen; the generation unit 207 is configured to, in response to detecting a selection operation by the user on any three-dimensional module in the second three-dimensional model displayed on the above interactive screen, generate a three-dimensional module information display page, where the above three-dimensional module information display page is a page for displaying the three-dimensional module information of the selected three-dimensional module; the second display unit 208 is configured to display the above three-dimensional module information display page on the above interactive screen.

[0107] It can be understood that the units described in the model display device 200 correspond to the respective steps in the method described with reference to Figure 1 . Thus, the operations, features, and beneficial effects described above for the method also apply to the model display device 200 and the units included therein, and will not be repeated here.

[0108] Next, with reference to Figure 3 , which shows a schematic structural diagram of an electronic device 300 suitable for implementing some embodiments of the present disclosure. The electronic devices in some embodiments of the present disclosure may include, but are not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Tablet Computers), PMPs (Portable Multimedia Players), vehicle terminals (such as vehicle navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc.Figure 3 The electronic device shown is merely an example and should not impose any limitations on the functions and scope of use of the embodiments of the present disclosure.

[0109] As Figure 3 shown, the electronic device 300 may include a processing device 301 (such as a central processing unit, a graphics processing unit, etc.), which may perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 302 or a program loaded from a storage device 308 into a random access memory (RAM) 303. In the RAM 303, various programs and data required for the operation of the electronic device 300 are also stored. The processing device 301, the ROM 302, and the RAM 303 are connected to each other through a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0110] Generally, the following devices may be connected to the I / O interface 305: an input device 306 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera group, a microphone, an accelerometer, a gyroscope, etc.; an output device 307 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 308 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 309. The communication device 309 may allow the electronic device 300 to communicate with other devices wirelessly or wiredly to exchange data. Although Figure 3 the electronic device 300 with various devices is shown, it should be understood that it is not required to implement or have all the shown devices. Instead, more or fewer devices may be implemented or had. Figure 3 Each block shown in

[0111] Specifically, according to some embodiments of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, some embodiments of the present disclosure include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes program codes for performing the methods shown in the flowcharts. In such some embodiments, the computer program may be downloaded and installed from a network through the communication device 309, or installed from the storage device 308, or installed from the ROM 302. When the computer program is executed by the processing device 301, the above functions defined in the methods of some embodiments of the present disclosure are performed.

[0112] It should be noted that the computer-readable media described in some embodiments of the present disclosure may be a computer-readable signal medium, a computer-readable storage medium, or any combination of the two. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In some embodiments of the present disclosure, the computer-readable storage medium may be any tangible medium that contains or stores a program, and the program can be used by or in conjunction with an instruction execution system, apparatus, or device. In some embodiments of the present disclosure, the computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer-readable signal medium may also be any computer-readable medium other than the computer-readable storage medium, and the computer-readable signal medium can send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted by any appropriate medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0113] In some embodiments, the client and the server can communicate using any currently known or future-developed network protocol such as HTTP (HyperText Transfer Protocol), and can be interconnected with digital data communication in any form or medium (e.g., a communication network). Examples of communication networks include local area networks ("LANs"), wide area networks ("WANs"), the Internet (e.g., the Internet), and end-to-end networks (e.g., ad hoc end-to-end networks), as well as any currently known or future-developed networks.

[0114] The above computer-readable medium may be included in the above electronic device; or it may exist independently without being assembled into the electronic device. The above computer-readable medium carries one or more programs. When the above one or more programs are executed by the electronic device, the electronic device is caused to: in response to detecting a startup operation for the interactive screen included in the above model interaction device, obtain a first three-dimensional model corresponding to the above two-dimensional drawing. According to the camera group included in the above interactive screen, collect the two-dimensional drawing corresponding to the above interactive screen to obtain a two-dimensional drawing image. Identify the target drawing identifier included in the above two-dimensional drawing image to obtain a drawing identifier recognition result. Determine the position information corresponding to the above target drawing identifier according to the target drawing identifier characterized by the above drawing identifier recognition result. According to the above position information and the interactive screen information of the above interactive screen, extract a second three-dimensional model corresponding to the above position information from the above first three-dimensional model. Display the above second three-dimensional model on the above interactive screen. In response to detecting a selection operation by the user on any three-dimensional module in the second three-dimensional model displayed on the above interactive screen, generate a three-dimensional module information display page. Display the above three-dimensional module information display page on the above interactive screen

[0115] Computer program code for performing the operations of some embodiments of the present disclosure may be written in one or more programming languages or combinations thereof. The above programming languages include object-oriented programming languages such as Java, Smalltalk, C++, and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, executed as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (for example, by using an Internet service provider to connect through the Internet).

[0116] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, a segment of code, or a part of code that contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions noted in the blocks may occur in a different order than noted in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, or they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and combinations of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system that performs the specified functions or operations, or by a combination of dedicated hardware and computer instructions.

[0117] The units described in some embodiments of the present disclosure can be implemented in software or in hardware. The described units can also be provided in a processor. For example, it can be described as: a processor includes an acquisition unit, a collection unit, an identification unit, a determination unit, an interception unit, a first display unit, a generation unit, and a second display unit. Among them, the names of these units do not constitute a limitation on the unit itself in some cases. For example, the acquisition unit can also be described as "the unit that acquires the first three-dimensional model corresponding to the above two-dimensional drawing in response to detecting a start operation for the interactive screen included in the above model interaction device".

[0118] The functions described above can be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that can be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on a chip (SOCs), complex programmable logic devices (CPLDs), and so on.

[0119] The above description is only some preferred embodiments of the present disclosure and an explanation of the technical principles applied. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by the specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above inventive concept. For example, the technical solutions formed by mutually replacing the above features with the (but not limited to) technical features having similar functions disclosed in the embodiments of the present disclosure.

Claims

1. A model display method, applied to a model interactive device, wherein the model interactive device comprises an interactive screen and a two-dimensional drawing, and the method comprises: In response to detecting a start-up operation for an interactive screen included in the model interactive device, obtaining a first three-dimensional model corresponding to the two-dimensional drawing, wherein the model interactive device further includes a sliding track, and the interactive screen slides along the track direction through the sliding track; According to the camera group included in the interactive screen, a two-dimensional drawing corresponding to the interactive screen is collected to obtain a two-dimensional drawing image; Identify the target drawing mark included in the two-dimensional drawing image to obtain a drawing mark identification result; Determine the position information corresponding to the target drawing identifier according to the target drawing identifier represented by the drawing identifier recognition result; Wherein, according to the target drawing mark represented by the drawing mark recognition result, determining the position information corresponding to the target drawing mark includes: Acquire interactive screen information and interactive screen display image of the interactive screen, wherein the interactive screen information includes the length and width of the interactive screen; Generate an interactive screen coordinate system according to the interactive screen display image; Determine the position information corresponding to the target drawing mark according to the interactive screen coordinate system, the interactive screen length and the interactive screen width; According to the position information and the interactive screen information of the interactive screen, extracting a second three-dimensional model corresponding to the position information from the first three-dimensional model; Displaying the second three-dimensional model on the interactive screen; In response to detecting a user's selection operation on any three-dimensional module in the second three-dimensional model displayed on the interactive screen, generating a three-dimensional module information display page, wherein the three-dimensional module information display page is a page for displaying the three-dimensional module information of the selected three-dimensional module; The three-dimensional module information display page is displayed on the interactive screen.

2. The method according to claim 1, wherein: The step of identifying the target drawing mark included in the two-dimensional drawing image to obtain a drawing mark identification result includes: Cutting each mark displayed on the two-dimensional drawing image to obtain a cut mark group; Each cutting mark in the cutting mark group is input into a pre-trained mark recognition model to obtain a drawing mark recognition result.

3. The method according to claim 2, wherein: The identification model is trained by the following steps: Acquire a sample set, wherein the samples in the sample set include a sample cutting mark and a sample drawing mark recognition result with the sample cutting mark; Selecting a sample from the sample set; Inputting the sample into the initial network model to obtain a drawing identification result corresponding to the sample; Determine a loss value between a drawing identification result corresponding to the sample and a sample drawing identification result included in the sample; In response to the loss value being greater than or equal to a preset threshold, adjusting the network parameters of the initial network model.

4. The method according to claim 3, wherein: The method further comprises: In response to the loss value being less than the preset threshold, the initial network model is determined as a logo recognition model.

5. The method according to claim 1, wherein: After collecting the two-dimensional drawing corresponding to the interactive screen according to the camera group included in the interactive screen to obtain the two-dimensional drawing image, the method further includes: Performing image segmentation processing on the two-dimensional drawing image to generate a segmented drawing image set; Acquire a preset three-dimensional module set from a target database, wherein the preset three-dimensional modules in the preset three-dimensional module set correspond to module numbers and module sizes; For each segmented drawing image in the segmented drawing image set, the following selection steps are performed: Selecting a preset three-dimensional module that meets a first preset condition from the preset three-dimensional module set as a target three-dimensional module; According to the position of the segmented drawing image in the two-dimensional drawing image, adjusting the scale and direction of the target three-dimensional module to obtain an adjusted three-dimensional module; According to the obtained adjusted three-dimensional modules, the two-dimensional drawing image is model reconstructed to generate a reconstructed three-dimensional model, and the reconstructed three-dimensional model is determined as the second three-dimensional model.

6. A model display device, comprising: an acquisition unit, configured to acquire a first three-dimensional model corresponding to the two-dimensional drawing in response to detecting a start-up operation for an interactive screen included in the model interaction device, wherein the model interaction device further includes a sliding track, and the interactive screen slides along the track direction through the sliding track; A collection unit is configured to collect the two-dimensional drawing corresponding to the interactive screen according to the camera group included in the interactive screen to obtain a two-dimensional drawing image; a recognition unit configured to recognize a target drawing mark included in the two-dimensional drawing image and obtain a drawing mark recognition result; A determination unit is configured to determine the position information corresponding to the target drawing identifier according to the target drawing identifier represented by the drawing identifier recognition result; the determination unit is further configured to: Acquire interactive screen information and interactive screen display image of the interactive screen, wherein the interactive screen information includes the length and width of the interactive screen; Generate an interactive screen coordinate system according to the interactive screen display image; Determine the position information corresponding to the target drawing mark according to the interactive screen coordinate system, the interactive screen length and the interactive screen width; a cutting unit configured to cut out a second three-dimensional model corresponding to the position information from the first three-dimensional model according to the position information and the interactive screen information of the interactive screen; A first display unit is configured to display the second three-dimensional model on the interactive screen; A generating unit is configured to generate a 3D module information display page in response to detecting a selection operation performed by a user on any 3D module in the second 3D model displayed on the interactive screen, wherein the 3D module information display page is a page for displaying 3D module information of the selected 3D module; The second display unit is configured to display the three-dimensional module information display page on the interactive screen.

7. An electronic device comprising: one or more processors; a storage device having one or more programs stored thereon; When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 5.

8. A computer readable medium having a computer program stored thereon, wherein: When the program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images