Data rendering method and system based on digital twins

Abstract

The present invention relates to a digital twin field, and discloses a data rendering method based on digital twins, including marking key areas of a first digital twin model based on first audio data of multiple areas, and rendering the first digital twin model as a second digital twin model based on the correlation between the motion change of the viewfinder and the key areas. The present invention uses the first audio data to mark the key areas of the first digital twin model on the basis of the first digital twin model generated by the traditional first video data, so as to give priority to rendering the key areas according to the areas and viewing trends that the user actually focuses on, thereby saving rendering computing power and improving the rendering effect perceived by the user.

Description

Technical Field

[0001] The present invention relates to the field of digital twins, and in particular to a data rendering method and system based on digital twins. Background Art

[0002] Digital twins make full use of data such as physical models, sensors, and operation history, integrate multi-disciplinary, multi-physical, multi-scale, and multi-probability simulation processes, and complete mapping in virtual space to reflect the entire life cycle of the corresponding physical equipment. Digital twins are a concept that transcends reality and can be viewed as a digital mapping system of one or more important, interdependent equipment systems.

[0003] In the production of industrial parks and factories, the management method of digital twins has gradually become a trend. It is rendered into a digital twin model through hardware for users to view.

[0004] Core elements of digital twin rendering:

[0005] 1. Data processing capability: The digital twin model involves a huge amount of data, including terrain, buildings, transportation, environment, etc. The rendering engine needs to have strong data processing capabilities to render these complex data in real time and accurately.

[0006] 2. Authenticity and accuracy: The rendering engine needs to restore the real world as realistically and accurately as possible while ensuring the rendering speed, which includes the detailed processing of lighting, materials, details, etc.

[0007] 3. Compatibility and scalability: Faced with ever-changing technologies and diverse application requirements, the rendering engine needs to have good compatibility and scalability to support new data formats and rendering technologies.

[0008] In terms of the current business model of property monitoring, monitoring hardware has been moving towards lower prices, but the rendering of digital twins consumes more computing power.

[0009] Therefore, there is a need for a data rendering method and system based on digital twins that can start from the user's actual perception as much as possible and increase the rendering effect on the user's perception without increasing the computing power pressure. Summary of the invention

[0010] The technical problem to be solved by the present invention is to provide a data rendering method and system based on digital twins, which can start from the user's actual perception as much as possible and increase the rendering effect on the user's perception without increasing the computing power pressure.

[0011] The present invention provides a data rendering method based on digital twins, comprising:

[0012] Acquire first video data and first audio data of multiple areas of the industrial park;

[0013] generating a first digital twin model based on the first video data of the plurality of regions;

[0014] Marking key areas of the first digital twin model based on the first audio data of the multiple areas, and rendering the first digital twin model into a second digital twin model based on the correlation between the motion change of the viewfinder and the key areas;

[0015] According to the second digital twin model, the second video data is output using the viewfinder.

[0016] The data rendering method based on digital twins of the present invention, wherein the key areas of the first digital twin model are marked based on the first audio data of multiple areas, including

[0017] Outputting decibel values ​​and text content per unit time according to the first audio data of the plurality of regions;

[0018] The membrane of the sound vector is configured according to the decibel value of the area corresponding to the first audio data; the corresponding pre-stored emotional angle is output according to the text content of the area corresponding to the first audio data; and the sound vector is constructed according to the emotional angle and the membrane of the vector;

[0019] Determine whether the height of the end point of the sound vector exceeds a first threshold. If so, configure the area of ​​the first audio data as a key area of ​​the first digital twin model.

[0020] The present invention is based on a data rendering method for digital twins, wherein, based on the correlation between the motion change of the view frame and the key area, the first digital twin model is rendered into a second digital twin model, including:

[0021] S410, in the key area of ​​the first digital twin model, obtaining the average value of the position of the center point of the view frame in the previous unit time in the first digital twin model as the first coordinate point; taking the center point of the current view frame and the position of the first digital twin model as the second coordinate point; constructing a trend vector from the first coordinate point to the second coordinate point; and configuring the end point of the trend vector extended by one time in the direction of the trend vector as the fifth coordinate point;

[0022] S420, taking the coordinate point farthest from the center point of the view frame of the previous unit time on one side of the trend vector as the third coordinate point, and the coordinate point farthest from the other side as the fourth coordinate point; emitting a first ray from the first coordinate point toward the third coordinate point; emitting a second ray from the first coordinate point toward the fourth coordinate point; emitting a third ray perpendicular to the trend vector passing through the second coordinate point; emitting a fourth ray perpendicular to the trend vector passing through the fifth coordinate point; configuring a triangular area enclosed by the first ray, the second ray, and the third ray as a first triangular area; configuring a trapezoidal area enclosed by the first ray, the second ray, the third ray, and the fourth ray as a second trapezoidal area;

[0023] S430. Construct a three-dimensional topographic map in the first digital twin model according to the height of the sound vector, configure the peak area of ​​the second trapezoidal area of ​​the three-dimensional topographic map as the first area; configure the peak area of ​​the first triangular area of ​​the three-dimensional topographic map as the second area; configure the peak area of ​​other areas of the three-dimensional topographic map as the third area; configure the middle area of ​​the second trapezoidal area of ​​the three-dimensional topographic map as the fourth area; configure the middle area of ​​the first triangular area of ​​the three-dimensional topographic map as the fifth area; configure the middle area of ​​other areas of the three-dimensional topographic map as the sixth area; configure the trough area of ​​the second trapezoidal area of ​​the three-dimensional topographic map as the seventh area; configure the trough area of ​​the first triangular area of ​​the three-dimensional topographic map as the eighth area; configure the trough area of ​​other areas of the three-dimensional topographic map as the ninth area;

[0024] S440, determining whether the proportion of the second trapezoidal area outside the first digital twin area exceeds a second threshold, if so, deleting the second trapezoidal area and jumping to S430; if not, jumping to S450;

[0025] S450, determining whether the proportion of the second trapezoidal area outside the first digital twin area exceeds a third threshold; if so, configuring the second trapezoidal area as a first triangular area and jumping to S430; if not, jumping to S460;

[0026] S460, determining whether the proportion of the first triangular area outside the first digital twin area exceeds a fourth threshold, if so, reducing the unit time and jumping to S410, if not, jumping to S470;

[0027] S470, configuring the correlation degrees in sequence according to the first area, the second area, the third area, the fourth area, the fifth area, the sixth area, the seventh area, the eighth area, and the ninth area, and rendering them in sequence;

[0028] S480: output the rendered first digital twin model as a second digital twin model.

[0029] The data rendering method of the present invention is based on digital twins, wherein the corresponding pre-stored emotional angle is output according to the text content of the area corresponding to the first audio data, including

[0030] Extracting a first keyword corresponding to the first word from the text content of the first audio data according to the first word and the preset angle corresponding to the first word pre-stored in the database;

[0031] Summing the preset angles corresponding to all the first keywords of the text content to obtain a first angle;

[0032] Output the emotion angle of the corresponding range interval according to the first angle.

[0033] The data rendering method of the present invention is based on digital twins, wherein the corresponding pre-stored emotional angle is output according to the text content of the area corresponding to the first audio data, including

[0034] Build an artificial intelligence assessment model;

[0035] Train AI rating models based on text content and corresponding emotional perspectives in historical data;

[0036] The text content corresponding to the first audio data is input into the artificial intelligence model to obtain the emotional angle corresponding to the text content.

[0037] The present invention is based on a data rendering method for digital twins, wherein a first digital twin model is rendered into a second digital twin model through parallel computing performed by a GPU.

[0038] The present invention is based on the data rendering method of digital twins, wherein the rendering engine is Unity or Unreal Engine.

[0039] In a second aspect, the present invention provides a data rendering system based on digital twins, comprising:

[0040] An input module, which is used to obtain first video data and first audio data of multiple areas of the industrial park;

[0041] A processor, configured to generate a first digital twin model based on the first video data of the multiple regions; mark key regions of the first digital twin model based on the first audio data of the multiple regions, and render the first digital twin model into a second digital twin model based on the correlation between the motion change of the viewfinder and the key regions;

[0042] An output module is used to output second video data using a viewfinder according to the second digital twin model.

[0043] The data rendering system of the present invention is based on digital twins, wherein the key areas of the first digital twin model are marked based on the first audio data of multiple areas, including

[0044] Outputting decibel values ​​and text content per unit time according to the first audio data of the plurality of regions;

[0045] The membrane of the sound vector is configured according to the decibel value of the area corresponding to the first audio data; the corresponding pre-stored emotional angle is output according to the text content of the area corresponding to the first audio data; and the sound vector is constructed according to the emotional angle and the membrane of the vector;

[0046] Determine whether the height of the end point of the sound vector exceeds a first threshold. If so, configure the area of ​​the first audio data as a key area of ​​the first digital twin model.

[0047] The present invention is based on a data rendering system of digital twins, wherein, based on the correlation between the motion change of the view frame and the key area, the first digital twin model is rendered into a second digital twin model, including

[0048] S410, in the key area of ​​the first digital twin model, obtaining the average value of the position of the center point of the view frame in the previous unit time in the first digital twin model as the first coordinate point; taking the center point of the current view frame and the position of the first digital twin model as the second coordinate point; constructing a trend vector from the first coordinate point to the second coordinate point; and configuring the end point of the trend vector extended by one time in the direction of the trend vector as the fifth coordinate point;

[0049] S420, taking the coordinate point farthest from the center point of the view frame of the previous unit time on one side of the trend vector as the third coordinate point, and the coordinate point farthest from the other side as the fourth coordinate point; emitting a first ray from the first coordinate point toward the third coordinate point; emitting a second ray from the first coordinate point toward the fourth coordinate point; emitting a third ray perpendicular to the trend vector passing through the second coordinate point; emitting a fourth ray perpendicular to the trend vector passing through the fifth coordinate point; configuring a triangular area enclosed by the first ray, the second ray, and the third ray as a first triangular area; configuring a trapezoidal area enclosed by the first ray, the second ray, the third ray, and the fourth ray as a second trapezoidal area;

[0050] S430. Construct a three-dimensional topographic map in the first digital twin model according to the height of the sound vector, configure the peak area of ​​the second trapezoidal area of ​​the three-dimensional topographic map as the first area; configure the peak area of ​​the first triangular area of ​​the three-dimensional topographic map as the second area; configure the peak area of ​​other areas of the three-dimensional topographic map as the third area; configure the middle area of ​​the second trapezoidal area of ​​the three-dimensional topographic map as the fourth area; configure the middle area of ​​the first triangular area of ​​the three-dimensional topographic map as the fifth area; configure the middle area of ​​other areas of the three-dimensional topographic map as the sixth area; configure the trough area of ​​the second trapezoidal area of ​​the three-dimensional topographic map as the seventh area; configure the trough area of ​​the first triangular area of ​​the three-dimensional topographic map as the eighth area; configure the trough area of ​​other areas of the three-dimensional topographic map as the ninth area;

[0051] S440, determining whether the proportion of the second trapezoidal area outside the first digital twin area exceeds a second threshold, if so, deleting the second trapezoidal area and jumping to S430; if not, jumping to S450;

[0052] S450, determining whether the proportion of the second trapezoidal area outside the first digital twin area exceeds a third threshold; if so, configuring the second trapezoidal area as a first triangular area and jumping to S430; if not, jumping to S460;

[0053] S460, determining whether the proportion of the first triangular area outside the first digital twin area exceeds a fourth threshold, if so, reducing the unit time and jumping to S410, if not, jumping to S470;

[0054] S470, configuring the correlation degrees in sequence according to the first area, the second area, the third area, the fourth area, the fifth area, the sixth area, the seventh area, the eighth area, and the ninth area, and rendering them in sequence;

[0055] S480: output the rendered first digital twin model as a second digital twin model.

[0056] The data rendering method based on digital twins of the present invention is different from the prior art in that the present invention uses the first audio data to mark the key areas of the first digital twin model on the basis of the first digital twin model generated by the traditional first video data, so as to know the areas that the user should focus on in the first digital twin model, and mark them as key areas, and then associate the above-mentioned key areas according to the movement changes of the viewfinder to adjust the rendering order and whether to render each area of ​​the first digital twin, so as to obtain the second digital twin model, and output the second video data based on it; thus, based on the areas that the user actually focuses on viewing and the viewing trends, the key areas are rendered first, so as to improve the rendering effect perceived by the user while saving rendering computing power.

[0057] The data rendering method based on digital twins of the present invention is further described below in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 It is a flow chart of the data rendering method based on digital twins;

[0059] Figure 2 is a flow chart of a method for generating a second digital twin model;

[0060] Figure 3 It is a schematic diagram of a three-dimensional topographic map;

[0061] Figure 4 This is a regional level diagram. DETAILED DESCRIPTION

[0062] First, as Figures 1 to 4 As shown, see Figure 1 The present invention provides a data rendering method based on digital twins, comprising:

[0063] Acquire first video data and first audio data of multiple areas of the industrial park;

[0064] generating a first digital twin model based on the first video data of the plurality of regions;

[0065] Marking key areas of the first digital twin model based on the first audio data of the multiple areas, and rendering the first digital twin model into a second digital twin model based on the correlation between the motion change of the viewfinder and the key areas;

[0066] According to the second digital twin model, the second video data is output using the viewfinder.

[0067] The present invention uses the first audio data to mark the key areas of the first digital twin model on the basis of the first digital twin model generated by the traditional first video data, so as to know the areas in the first digital twin model that the user should focus on, and mark them as key areas, and then associate the above-mentioned key areas according to the movement changes of the viewfinder to adjust the rendering order and whether to render each area of ​​the first digital twin, so as to obtain the second digital twin model, and output the second video data accordingly; thus, based on the areas that the user actually focuses on viewing and the viewing trends, the key areas are rendered first, so as to improve the rendering effect perceived by the user while saving rendering computing power.

[0068] Each area of ​​the industrial park can be Figure 4 In each of the grids, each area can be configured with a camera and a microphone, which are used to collect the first video data and the first audio data, and summarize them to the processor, and each area can be processed separately.

[0069] Among them, the first digital twin model is generated according to the first video data of the multiple areas. It can be understood that: the present invention can use the existing technology to synthesize the video data of each area into a first digital twin model. It should be noted that compared with the second digital twin model, the first digital twin model has lower clarity and lower rendering degree. It can be used as a minimum rendered digital twin model to facilitate improving the frame rate and reducing computing pressure.

[0070] The second digital twin model can be understood as a digital twin model with better partial or overall rendering. In other words, assuming that the viewfinder controlled by the user stays for a longer time, the second digital twin model is a model in which the first digital twin model is fully rendered in the order of the first area to the ninth area. Assuming that the viewfinder controlled by the user stays for a shorter time, what the user may see is that the first digital twin model has only rendered a part of the model, which is rendered in more important areas, so that the user has rendered the position that may be observed, but not too much rendering is performed in the position that the user may not pay too much attention to, thereby improving the user's sensory experience without increasing the computing pressure.

[0071] Among them, according to the second digital twin model, the second video data is output using the viewfinder, which can be understood as: the second digital twin model is used as the model we rendered, and the second digital twin model in the viewfinder can be used as the second video data of the video image that the user can actually watch.

[0072] In some embodiments, see Figure 1 , the first audio data based on the multiple regions are used to mark the key regions of the first digital twin model, including

[0073] Outputting decibel values ​​and text content per unit time according to the first audio data of the plurality of regions;

[0074] The membrane of the sound vector is configured according to the decibel value of the area corresponding to the first audio data; the corresponding pre-stored emotional angle is output according to the text content of the area corresponding to the first audio data; and the sound vector is constructed according to the emotional angle and the membrane of the vector;

[0075] Determine whether the height of the end point of the sound vector exceeds a first threshold. If so, configure the area of ​​the first audio data as a key area of ​​the first digital twin model.

[0076] The present invention constructs a sound vector by quantitatively splitting the above-mentioned first audio data into membranes corresponding to decibel values ​​and emotional angles corresponding to textual content, and judges the height of the end point of the sound vector, thereby affecting the height of the end point of the sound vector, and then judges whether it is the end point area based on this height, so as to make a comprehensive judgment on the degree of the end point of this area from the decibel value and textual content.

[0077] Among them, according to the first audio data of multiple areas, the decibel value and text content in the unit time are output, which can be understood as: the maximum decibel value of the first audio data of one area in the unit time is the decibel value in the unit time. That is, the above decibel value may be a situation where people call for help, shout, explosion, metal breakage, and alarm sound. The larger the decibel value, the more it represents the location that users who observe the digital twin model may notice. Then the sound vector it generates is naturally longer in length and its vertical height is naturally higher, which is bound to form a peak on the three-dimensional map, so this area should be rendered first.

[0078] The text content of the first audio data in one area per unit time can be: the specific content of the first audio data is converted into text content through voice-to-text software, and the specific conversation content of the user is easily recognized. If someone is shouting for help, or something has happened, or someone has died, then the emotional angle it brings must be more vertical, that is, closer to 90 degrees, which can maximize the height of the sound vector to the highest extent and form a peak. This area should be rendered first. From an emotional perspective, the user's text content can have many directional keywords, such as: It's okay, which can make the emotional angle more horizontal, thereby reducing the maximum height of the sound vector; such as: It's okay, it's okay, this kind of multiple negative keywords can make the emotional angle more horizontal.

[0079] Among them, the membrane of the sound vector is configured according to the decibel value of the area corresponding to the first audio data. It can be understood that: the larger the above decibel value is, the longer the sound vector is, and the higher the height of the sound vector is.

[0080] The sound vector can be along 40 decibels to 120 decibels, and the membrane with uniform sound vector configuration is 40cm~120cm. Among them, the decibel value below 40 decibels can generate a sound vector, which represents a trough.

[0081] Among them, the pre-stored emotional angle corresponding to the text content of the area corresponding to the first audio data is outputted, which can be understood as: the user can query the corresponding angle of each word stored in the database, and sum them to obtain the emotional angle of the first audio data of this unit time; specifically, for example, when no keywords appear, the emotional angle is the initial value of 45 degrees. For example, positive words stored in the database are: help, corresponding to 90 degrees; fire, corresponding to 90 degrees; too hot, corresponding to 20 degrees; very, very hot, corresponding to 30 degrees; negative words are: it's okay, corresponding to -10 degrees, it's okay, it's okay, corresponding to -20 degrees; specific words can be preset in the database.

[0082] Finally, add up all the words, and the lowest emotional angle should not be lower than 0 degrees and the highest should not be higher than 90 degrees.

[0083] Among them, the sound vector is constructed according to the emotional angle and the membrane of the vector, including: taking the decibel value as the length of the vector, that is, the membrane of the vector; taking the emotional angle as the tilt angle of the vector. The center position of the XY axis coordinates of each area is taken as the starting position of the emotional vector. The Z value of the sound vector is the height of the vector. Then when constructing a three-dimensional topographic map, the height of this area is the above Z value, and the XY coordinate of this Z value is the center position of the XY coordinates of this area.

[0084] Among them, judging whether the height of the end point of the sound vector exceeds the first threshold value is to judge whether the above Z value exceeds the first threshold value.

[0085] The first threshold may be 52 cm, for example, when the emotional angle is 60 degrees and the upper limit of human dialogue is 60 decibels, the height of the end point of the sound vector is approximately 52 cm. If it exceeds this, it should be paid special attention.

[0086] In some embodiments, Figure 2 , 3 , 4, and based on the correlation between the motion change of the view frame and the key area, rendering the first digital twin model into a second digital twin model, including

[0087] S410, in the key area of ​​the first digital twin model, obtaining the average value of the position of the center point of the view frame in the previous unit time in the first digital twin model as the first coordinate point; taking the center point of the current view frame and the position of the first digital twin model as the second coordinate point; constructing a trend vector from the first coordinate point to the second coordinate point; and configuring the end point of the trend vector extended by one time in the direction of the trend vector as the fifth coordinate point;

[0088] S420, taking the coordinate point farthest from the center point of the view frame of the previous unit time on one side of the trend vector as the third coordinate point, and the coordinate point farthest from the other side as the fourth coordinate point; emitting a first ray from the first coordinate point toward the third coordinate point; emitting a second ray from the first coordinate point toward the fourth coordinate point; emitting a third ray perpendicular to the trend vector passing through the second coordinate point; emitting a fourth ray perpendicular to the trend vector passing through the fifth coordinate point; configuring a triangular area enclosed by the first ray, the second ray, and the third ray as a first triangular area; configuring a trapezoidal area enclosed by the first ray, the second ray, the third ray, and the fourth ray as a second trapezoidal area;

[0089] S430. Construct a three-dimensional topographic map in the first digital twin model according to the height of the sound vector, configure the peak area of ​​the second trapezoidal area of ​​the three-dimensional topographic map as the first area; configure the peak area of ​​the first triangular area of ​​the three-dimensional topographic map as the second area; configure the peak area of ​​other areas of the three-dimensional topographic map as the third area; configure the middle area of ​​the second trapezoidal area of ​​the three-dimensional topographic map as the fourth area; configure the middle area of ​​the first triangular area of ​​the three-dimensional topographic map as the fifth area; configure the middle area of ​​other areas of the three-dimensional topographic map as the sixth area; configure the trough area of ​​the second trapezoidal area of ​​the three-dimensional topographic map as the seventh area; configure the trough area of ​​the first triangular area of ​​the three-dimensional topographic map as the eighth area; configure the trough area of ​​other areas of the three-dimensional topographic map as the ninth area;

[0090] S440, determining whether the proportion of the second trapezoidal area outside the first digital twin area exceeds a second threshold, if so, deleting the second trapezoidal area and jumping to S430; if not, jumping to S450;

[0091] S450, determining whether the proportion of the second trapezoidal area outside the first digital twin area exceeds a third threshold; if so, configuring the second trapezoidal area as a first triangular area and jumping to S430; if not, jumping to S460;

[0092] S460, determining whether the proportion of the first triangular area outside the first digital twin area exceeds a fourth threshold, if so, reducing the unit time and jumping to S410, if not, jumping to S470;

[0093] S470, configuring the correlation degrees in sequence according to the first area, the second area, the third area, the fourth area, the fifth area, the sixth area, the seventh area, the eighth area, and the ninth area, and rendering them in sequence;

[0094] S480: output the rendered first digital twin model as a second digital twin model.

[0095] The present invention first segments the motion trajectory of the viewfinder according to the initial unit time, and uses the previous unit time at this time as the motion status of the viewfinder and the coordinates of the center point of the viewfinder at this time as the situation for judging the trend of the viewfinder, and renders the second trapezoidal area representing the upcoming trend of the viewfinder as a priority, and renders the area of ​​the viewfinder of the previous unit time as the second, and other areas as the last; and, according to the peaks and troughs of the three-dimensional terrain area constructed according to the height of the sound vector, the location area where the key sound is emitted can be found, and the peak area of ​​the three-dimensional terrain map is focused on as the priority rendering area. At this point, the present invention integrates the correlation between the sound vector and the motion change of the viewfinder, and uses this correlation to render each area one by one, so that when the current computing power cannot directly render the first digital twin model completely, the key areas are rendered first, and then the non-key areas are rendered according to the order and key areas that the user may observe, thereby improving the user-perceived clarity of the output second digital twin model, and reducing the computing power pressure to a certain extent, and improving the frame rate of the output second video data when using lower computing power.

[0096] Among them, in the key area of ​​the first digital twin model, the average value of the position of the center point of the view frame in the previous unit time in the first digital twin model is obtained as the first coordinate point, which can be understood as: Figure 2 , 3 4. The key area selected by the first threshold is used as a reference object. In the key area, the motion trajectory of the previous unit time is as follows: Figure 4 , and generate the first coordinate point according to its motion trajectory and its weight in each second of the first unit time. For example, if the first unit time is 3 seconds, the x value of the first 2 seconds is 3, and the x value of the third second is 6, then the final x value of the first coordinate point is 4. The y value of the first coordinate point is the same. It should be noted that the Z value does not need to be considered in this step, that is, the analysis and processing are all performed in the XY coordinates.

[0097] Among them, the center point of the current viewfinder and the position of the first digital twin model are used as the second coordinate point, that is, along the direction of the viewfinder, the first vector is constructed with the center point of the viewfinder, and the intersection of this first vector and the plane XY coordinate system of the bottom surface of the first digital twin model is the second coordinate point;

[0098] Among them, the end point where the trend vector is extended twice in the direction of the trend vector is configured as the fifth coordinate point, which can be understood as: this fifth coordinate point can be understood as we believe that at the end of this current unit time, we roughly follow the trend of the viewfinder and the second coordinate point may fall into the fifth coordinate point. The fifth coordinate point represents a trend point.

[0099] The first triangular area and the second trapezoidal area refer to Figure 4 .

[0100] Among them, a three-dimensional topographic map is constructed in the first digital twin model according to the height of the sound vector. It can be understood that the height of the sound vector of each area is taken as the Z point, and the center point of the XY coordinate system of the area at this height point is taken as the XY value of this Z point, that is, the highest point of each area appears at the center of this area, and the highest points of these areas form a smooth three-dimensional topographic map. The peaks of this three-dimensional topographic map can reflect the area where the sound vector is the highest, and the troughs are the opposite.

[0101] In this case, the three-dimensional topographic map can consider only the key areas of the first digital twin model, or all areas.

[0102] In the process of configuring the first to ninth areas, it is easy for one area to be configured as two areas at the same time. Figure 4 , Figure 4 1 represents the first area, 2 represents the second area, and so on. The same area is configured as the first area and the third area; the same area is configured as the first area and the second area; the same area is configured as the fourth area and the fifth area. Then, we should select a smaller area to define this area, that is, the core concept is to give priority to rendering the area that the user may pay attention to, and render it completely. In other words, if the same area is configured as the first area and the third area, it should be configured as the first area.

[0103] Among them, it is determined whether the proportion of the second trapezoidal area outside the first digital twin area exceeds the second threshold. If so, the second trapezoidal area is deleted and jumps to S460. It can be understood that if the proportion of the second trapezoidal area located in the first digital twin area is too large, then this second trapezoidal area should not be used as the user's viewing trend position and should be rendered with priority. For example, the second threshold may be 70%. If 70% of the area of ​​the second trapezoidal area is located outside the first digital twin area, then the second trapezoidal area can be deleted. That is, if the user's future viewing trend is not this second trapezoidal area, then the rendering priority corresponding to this second trapezoidal area will not be increased, or even reduced.

[0104] Among them, it is determined whether the proportion of the second trapezoidal area outside the first digital twin area exceeds the third threshold. If it exceeds, the second trapezoidal area is configured as the first triangular area and jumps to S430. It can be understood as: if a part of the current second trapezoidal area is outside the first digital twin area, but not excessively, the second trapezoidal area and the first triangular area can be merged into the first triangular area, and the level of each area can be redefined, that is, if there is a part of the trapezoidal area outside, it is not a trend that users are likely to watch, so the rendering priority of the above area is neither increased nor decreased. Among them, the third threshold can be 20%, that is, when more than 20% of the second trapezoidal area is located outside the first digital twin area, the second trapezoidal area is merged with the first triangular area and rendered with the same priority.

[0105] Among them, it is determined whether the proportion of the first triangular area outside the first digital twin area exceeds the fourth threshold. If so, the unit time is reduced and the process jumps to S410. It can be understood that if most of the user's first triangular area is outside the first unit time, an error value may appear. This error value may be caused by frequent activation of the viewfinder or frequent viewing of the user outside the first digital twin area. This does not have a certain degree of emphasis and trend on the current rendering priority. Therefore, we can reduce the unit time and flush the data generated by a shorter unit time that is closer to the current time and may exclude the error value, and make a new judgment.

[0106] In some embodiments, Figure 2 , 3 4. Outputting the corresponding pre-stored emotional angle according to the text content of the area corresponding to the first audio data, including

[0107] Extracting a first keyword corresponding to the first word from the text content of the first audio data according to the first word and the preset angle corresponding to the first word pre-stored in the database;

[0108] Summing the preset angles corresponding to all the first keywords of the text content to obtain a first angle;

[0109] Output the emotion angle of the corresponding range interval according to the first angle.

[0110] The present invention extracts each first keyword in the file content and calculates the corresponding preset angle, thereby evaluating the importance of the entire text content and outputting the corresponding emotional angle. This angle can represent whether the user is likely to focus on this area and form a sound vector that is more in line with the user's attention.

[0111] In some embodiments, Figure 2 , 3 4. Outputting the corresponding pre-stored emotional angle according to the text content of the area corresponding to the first audio data, including

[0112] Build an artificial intelligence assessment model;

[0113] Train AI rating models based on text content and corresponding emotional perspectives in historical data;

[0114] The text content corresponding to the first audio data is input into the artificial intelligence model to obtain the emotional angle corresponding to the text content.

[0115] The present invention uses the artificial intelligence assessment model, a large model technology, to output the corresponding emotional angle for the text content, wherein the artificial intelligence assessment model is based on the emotional angle corresponding to the text content already obtained in the historical data, which will make the emotional angle evaluated by the artificial intelligence assessment model more consistent with the historical data.

[0116] In some embodiments, Figure 2 , 3 4. Render the first digital twin model into a second digital twin model through parallel computing performed by GPU.

[0117] The present invention can improve rendering performance through parallel computing performed by the GPU. Of course, the present invention can also obtain the second digital twin model by directly performing sequential computing through a high-performance GPU instead of parallel computing of the GPU.

[0118] In some embodiments, Figure 2 , 3 4. The rendering engine used for the rendering is Unity or Unreal Engine.

[0119] Second, as Figure 1 As shown, referring to the present invention, a data rendering system based on digital twins is characterized by: comprising

[0120] An input module, which is used to obtain first video data and first audio data of multiple areas of the industrial park;

[0121] A processor, configured to generate a first digital twin model based on the first video data of the multiple regions; mark key regions of the first digital twin model based on the first audio data of the multiple regions, and render the first digital twin model into a second digital twin model based on the correlation between the motion change of the viewfinder and the key regions;

[0122] An output module is used to output second video data using a viewfinder according to the second digital twin model.

[0123] The present invention uses the first audio data to mark the key areas of the first digital twin model on the basis of the first digital twin model generated by the traditional first video data, so as to know the areas in the first digital twin model that the user should focus on, and mark them as key areas, and then associate the above-mentioned key areas according to the movement changes of the viewfinder to adjust the rendering order and whether to render each area of ​​the first digital twin, so as to obtain the second digital twin model, and output the second video data accordingly; thus, based on the areas that the user actually focuses on viewing and the viewing trends, the key areas are rendered first, so as to improve the rendering effect perceived by the user while saving rendering computing power.

[0124] In some embodiments, Figure 2 , 3 4. The step of marking the key areas of the first digital twin model based on the first audio data of the plurality of areas includes:

[0125] Outputting decibel values ​​and text content per unit time according to the first audio data of the plurality of regions;

[0126] The membrane of the sound vector is configured according to the decibel value of the area corresponding to the first audio data; the corresponding pre-stored emotional angle is output according to the text content of the area corresponding to the first audio data; and the sound vector is constructed according to the emotional angle and the membrane of the vector;

[0127] Determine whether the height of the end point of the sound vector exceeds a first threshold. If so, configure the area of ​​the first audio data as a key area of ​​the first digital twin model.

[0128] The present invention constructs a sound vector by quantitatively splitting the above-mentioned first audio data into membranes corresponding to decibel values ​​and emotional angles corresponding to textual content, and judges the height of the end point of the sound vector, thereby affecting the height of the end point of the sound vector, and then judges whether it is the end point area based on this height, so as to make a comprehensive judgment on the degree of the end point of this area from the decibel value and textual content.

[0129] In some embodiments, Figure 2 , 3 , 4, and based on the correlation between the motion change of the view frame and the key area, rendering the first digital twin model into a second digital twin model, including

[0130] S410, in the key area of ​​the first digital twin model, obtaining the average value of the position of the center point of the view frame in the previous unit time in the first digital twin model as the first coordinate point; taking the center point of the current view frame and the position of the first digital twin model as the second coordinate point; constructing a trend vector from the first coordinate point to the second coordinate point; and configuring the end point of the trend vector extended by one time in the direction of the trend vector as the fifth coordinate point;

[0131] S420, taking the coordinate point farthest from the center point of the view frame of the previous unit time on one side of the trend vector as the third coordinate point, and the coordinate point farthest from the other side as the fourth coordinate point; emitting a first ray from the first coordinate point toward the third coordinate point; emitting a second ray from the first coordinate point toward the fourth coordinate point; emitting a third ray perpendicular to the trend vector passing through the second coordinate point; emitting a fourth ray perpendicular to the trend vector passing through the fifth coordinate point; configuring a triangular area enclosed by the first ray, the second ray, and the third ray as a first triangular area; configuring a trapezoidal area enclosed by the first ray, the second ray, the third ray, and the fourth ray as a second trapezoidal area;

[0132] S430. Construct a three-dimensional topographic map in the first digital twin model according to the height of the sound vector, configure the peak area of ​​the second trapezoidal area of ​​the three-dimensional topographic map as the first area; configure the peak area of ​​the first triangular area of ​​the three-dimensional topographic map as the second area; configure the peak area of ​​other areas of the three-dimensional topographic map as the third area; configure the middle area of ​​the second trapezoidal area of ​​the three-dimensional topographic map as the fourth area; configure the middle area of ​​the first triangular area of ​​the three-dimensional topographic map as the fifth area; configure the middle area of ​​other areas of the three-dimensional topographic map as the sixth area; configure the trough area of ​​the second trapezoidal area of ​​the three-dimensional topographic map as the seventh area; configure the trough area of ​​the first triangular area of ​​the three-dimensional topographic map as the eighth area; configure the trough area of ​​other areas of the three-dimensional topographic map as the ninth area;

[0133] S440, determining whether the proportion of the second trapezoidal area outside the first digital twin area exceeds a second threshold, if so, deleting the second trapezoidal area and jumping to S430; if not, jumping to S450;

[0134] S450, determining whether the proportion of the second trapezoidal area outside the first digital twin area exceeds a third threshold; if so, configuring the second trapezoidal area as a first triangular area and jumping to S430; if not, jumping to S460;

[0135] S460, determining whether the proportion of the first triangular area outside the first digital twin area exceeds a fourth threshold, if so, reducing the unit time and jumping to S410, if not, jumping to S470;

[0136] S470, configuring the correlation degrees in sequence according to the first area, the second area, the third area, the fourth area, the fifth area, the sixth area, the seventh area, the eighth area, and the ninth area, and rendering them in sequence;

[0137] S480: output the rendered first digital twin model as a second digital twin model.

[0138] The present invention first segments the motion trajectory of the viewfinder according to the initial unit time, and uses the previous unit time at this time as the motion status of the viewfinder and the coordinates of the center point of the viewfinder at this time as the situation for judging the trend of the viewfinder, and renders the second trapezoidal area representing the upcoming trend of the viewfinder as a priority, and renders the area of ​​the viewfinder of the previous unit time as the second, and other areas as the last; and, according to the peaks and troughs of the three-dimensional terrain area constructed according to the height of the sound vector, the location area where the key sound is emitted can be found, and the peak area of ​​the three-dimensional terrain map is focused on as the priority rendering area. At this point, the present invention integrates the correlation between the sound vector and the motion change of the viewfinder, and uses this correlation to render each area one by one, so that when the current computing power cannot directly render the first digital twin model completely, the key areas are rendered first, and then the non-key areas are rendered according to the order and key areas that the user may observe, thereby improving the user-perceived clarity of the output second digital twin model, and reducing the computing power pressure to a certain extent, and improving the frame rate of the output second video data when using lower computing power.

[0139] The embodiments described above are merely descriptions of preferred implementation modes of the present invention and are not intended to limit the scope of the present invention. Without departing from the design spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary technicians in this field should all fall within the protection scope determined by the claims of the present invention.

Claims

1. A data rendering method based on digital twins, characterized in that: include Acquire first video data and first audio data of multiple areas of the industrial park; generating a first digital twin model based on the first video data of the plurality of regions; Marking key areas of the first digital twin model based on the first audio data of the multiple areas, and rendering the first digital twin model into a second digital twin model based on the correlation between the motion change of the viewfinder and the key areas; Outputting second video data using a viewfinder according to the second digital twin model; The marking of the key areas of the first digital twin model based on the first audio data of the plurality of areas includes: Outputting decibel values ​​and text content per unit time according to the first audio data of the plurality of regions; configuring a membrane of a sound vector according to a decibel value of a region corresponding to the first audio data; Outputting a corresponding pre-stored emotional angle according to the text content of the area corresponding to the first audio data; constructing a sound vector according to the emotional angle and the vector membrane; Determine whether the height of the end point of the sound vector exceeds a first threshold, and if so, configure the area of ​​the first audio data as a key area of ​​the first digital twin model; Based on the correlation between the motion change of the view frame and the key area, the first digital twin model is rendered into a second digital twin model, including S410, in the key area of ​​the first digital twin model, obtaining the average value of the position of the center point of the view frame in the previous unit time in the first digital twin model as the first coordinate point; taking the center point of the current view frame and the position of the first digital twin model as the second coordinate point; constructing a trend vector from the first coordinate point to the second coordinate point; Configure the end point where the trend vector is extended by twice in the direction of the trend vector as the fifth coordinate point; S420, taking the coordinate point farthest from the center point of the view frame of the previous unit time on one side of the trend vector as the third coordinate point, and the coordinate point farthest from the other side as the fourth coordinate point; emitting a first ray from the first coordinate point toward the third coordinate point; emitting a second ray from the first coordinate point toward the fourth coordinate point; emitting a third ray perpendicular to the trend vector passing through the second coordinate point; emitting a fourth ray perpendicular to the trend vector passing through the fifth coordinate point; configuring a triangular area enclosed by the first ray, the second ray, and the third ray as a first triangular area; configuring a trapezoidal area enclosed by the first ray, the second ray, the third ray, and the fourth ray as a second trapezoidal area; S430, constructing a three-dimensional topographic map in the first digital twin model according to the height of the sound vector, and configuring a peak area of ​​a second trapezoidal area of ​​the three-dimensional topographic map as a first area; The peak area of ​​the first triangular area of ​​the three-dimensional topographic map is configured as the second area; the peak area of ​​the other areas of the three-dimensional topographic map is configured as the third area; the middle area of ​​the second trapezoidal area of ​​the three-dimensional topographic map is configured as the fourth area; the middle area of ​​the first triangular area of ​​the three-dimensional topographic map is configured as the fifth area; the middle area of ​​the other areas of the three-dimensional topographic map is configured as the sixth area; the trough area of ​​the second trapezoidal area of ​​the three-dimensional topographic map is configured as the seventh area; configuring the trough area of ​​the first triangular area of ​​the three-dimensional topographic map as the eighth area; configuring the trough area of ​​other areas of the three-dimensional topographic map as the ninth area; S440, determining whether the proportion of the second trapezoidal area outside the first digital twin area exceeds a second threshold, if so, deleting the second trapezoidal area and jumping to S430; if not, jumping to S450; S450, determining whether the proportion of the second trapezoidal area outside the first digital twin area exceeds a third threshold; if so, configuring the second trapezoidal area as a first triangular area and jumping to S430; if not, jumping to S460; S460, determining whether the proportion of the first triangular area outside the first digital twin area exceeds a fourth threshold, if so, reducing the unit time and jumping to S410, if not, jumping to S470; S470, configuring the correlation degrees in sequence according to the first area, the second area, the third area, the fourth area, the fifth area, the sixth area, the seventh area, the eighth area, and the ninth area, and rendering them in sequence; S480: output the rendered first digital twin model as a second digital twin model.

2. The data rendering method based on digital twins according to claim 1 is characterized in that: Outputting the corresponding pre-stored emotional angle according to the text content of the area corresponding to the first audio data, including Extracting a first keyword corresponding to the first word from the text content of the first audio data according to the first word and the preset angle corresponding to the first word pre-stored in the database; Summing the preset angles corresponding to all the first keywords of the text content to obtain a first angle; Output the emotion angle of the corresponding range interval according to the first angle.

3. The data rendering method based on digital twin according to claim 1, characterized in that: Outputting the corresponding pre-stored emotional angle according to the text content of the area corresponding to the first audio data, including Build an artificial intelligence assessment model; Train AI rating models based on text content and corresponding emotional perspectives in historical data; The text content corresponding to the first audio data is input into the artificial intelligence model to obtain the emotional angle corresponding to the text content.

4. The data rendering method based on digital twins according to claim 1 is characterized in that: The first digital twin model is rendered into a second digital twin model through parallel computing performed by a GPU.

5. The data rendering method based on digital twin according to claim 1, characterized in that: The rendering engine used for the rendering is Unity or Unreal Engine.

6. A data rendering system based on digital twins, characterized by: include An input module, which is used to obtain first video data and first audio data of multiple areas of the industrial park; A processor, configured to generate a first digital twin model according to the first video data of the plurality of regions; mark key regions of the first digital twin model based on the first audio data of the plurality of regions, and render the first digital twin model into a second digital twin model based on a correlation between a motion change of a framing frame and the key regions; An output module, configured to output second video data using a viewfinder according to the second digital twin model; The marking of the key areas of the first digital twin model based on the first audio data of the plurality of areas includes: Outputting decibel values ​​and text content per unit time according to the first audio data of the plurality of regions; configuring a membrane of a sound vector according to a decibel value of a region corresponding to the first audio data; Outputting a corresponding pre-stored emotional angle according to the text content of the area corresponding to the first audio data; constructing a sound vector according to the emotional angle and the vector membrane; Determine whether the height of the end point of the sound vector exceeds a first threshold, and if so, configure the area of ​​the first audio data as a key area of ​​the first digital twin model; Based on the correlation between the motion change of the view frame and the key area, the first digital twin model is rendered into a second digital twin model, including S410, in the key area of ​​the first digital twin model, obtaining the average value of the position of the center point of the view frame in the previous unit time in the first digital twin model as the first coordinate point; taking the center point of the current view frame and the position of the first digital twin model as the second coordinate point; constructing a trend vector from the first coordinate point to the second coordinate point; Configure the end point where the trend vector is extended by twice in the direction of the trend vector as the fifth coordinate point; S420, taking the coordinate point farthest from the center point of the view frame of the previous unit time on one side of the trend vector as the third coordinate point, and the coordinate point farthest from the other side as the fourth coordinate point; emitting a first ray from the first coordinate point toward the third coordinate point; emitting a second ray from the first coordinate point toward the fourth coordinate point; emitting a third ray perpendicular to the trend vector passing through the second coordinate point; emitting a fourth ray perpendicular to the trend vector passing through the fifth coordinate point; configuring a triangular area enclosed by the first ray, the second ray, and the third ray as a first triangular area; configuring a trapezoidal area enclosed by the first ray, the second ray, the third ray, and the fourth ray as a second trapezoidal area; S430, constructing a three-dimensional topographic map in the first digital twin model according to the height of the sound vector, and configuring a peak area of ​​a second trapezoidal area of ​​the three-dimensional topographic map as a first area; The peak area of ​​the first triangular area of ​​the three-dimensional topographic map is configured as the second area; the peak area of ​​the other areas of the three-dimensional topographic map is configured as the third area; the middle area of ​​the second trapezoidal area of ​​the three-dimensional topographic map is configured as the fourth area; the middle area of ​​the first triangular area of ​​the three-dimensional topographic map is configured as the fifth area; the middle area of ​​the other areas of the three-dimensional topographic map is configured as the sixth area; the trough area of ​​the second trapezoidal area of ​​the three-dimensional topographic map is configured as the seventh area; configuring the trough area of ​​the first triangular area of ​​the three-dimensional topographic map as the eighth area; configuring the trough area of ​​other areas of the three-dimensional topographic map as the ninth area; S440, determining whether the proportion of the second trapezoidal area outside the first digital twin area exceeds a second threshold, if so, deleting the second trapezoidal area and jumping to S430; if not, jumping to S450; S450, determining whether the proportion of the second trapezoidal area outside the first digital twin area exceeds a third threshold; if so, configuring the second trapezoidal area as a first triangular area and jumping to S430; if not, jumping to S460; S460, determining whether the proportion of the first triangular area outside the first digital twin area exceeds a fourth threshold, if so, reducing the unit time and jumping to S410, if not, jumping to S470; S470, configuring the correlation degrees in sequence according to the first area, the second area, the third area, the fourth area, the fifth area, the sixth area, the seventh area, the eighth area, and the ninth area, and rendering them in sequence; S480: output the rendered first digital twin model as a second digital twin model.

Images