Production line modeling method based on digital twinning and electronic equipment

Abstract

The embodiment of the invention discloses a production line modeling method based on digital twinning and electronic equipment, and the method comprises the steps: collecting environment data around an automatic guided vehicle in a process of controlling the automatic guided vehicle to move on a production line; the environment data at least comprises depth data and position data of each object on the production line; based on the depth data, the position data and a preset condition, determining a passable channel and a passage route and passage parameters of the automated guided vehicle; determining the position and shape of the object based on the depth data and the position data; and controlling the automatic guided vehicle to move according to the passing route and the passing parameters, and establishing a production line model of the production line based on the position and the shape of the object.

Description

Technical Field

[0001] The present application relates to the field of production and manufacturing technology, and in particular to a production line modeling method and electronic equipment based on digital twins. Background Art

[0002] Digital Twin is a technology that digitizes physical entities or systems in the real world to create corresponding models in virtual space. Using digital twins to build production line models can monitor the status of the production line in real time and help operators understand the operation of the production line in a timely manner.

[0003] The existing production line modeling method uses 3D modeling technology to model the production line. However, this modeling method can only reflect the static production line and cannot reflect the movement or replacement of items in the production line. Therefore, the existing production line modeling method has the problems of single functionality and poor real-time performance. Summary of the invention

[0004] In order to solve the above technical problems, the embodiments of the present application hope to provide a production line modeling method and electronic equipment based on digital twins.

[0005] The technical solution of this application is implemented as follows:

[0006] A production line modeling method based on digital twins, the method comprising: collecting environmental data around the automatic guided vehicle while controlling the automatic guided vehicle to move on the production line; the environmental data at least comprising: depth data and position data of each object on the production line; determining a navigable channel, and a travel route and travel parameters of the automatic guided vehicle based on the depth data, the position data and preset conditions; determining the position and shape of the object based on the depth data and the position data; controlling the automatic guided vehicle to move according to the travel route and the travel parameters, and establishing a production line model of the production line based on the position and shape of the object.

[0007] In the above scheme, based on the depth data, position data and preset conditions, the passable channel, as well as the pass route and pass parameters of the automatic guided vehicle are determined, including: based on the depth data, position data and the first preset condition, the first target search area is determined; the first preset condition is the method for determining the first target search area; based on the distance difference between each two adjacent first target search areas and the second preset condition, the passable channel is determined; the second preset condition is the method for determining the passable channel; based on the passable channel, the pass route and pass parameters are determined.

[0008] A production line modeling device based on digital twins, the device comprising: a first control module, used to control an automatic guided vehicle to collect environmental data around the automatic guided vehicle during its movement on the production line; the environmental data at least includes: depth data and position data of each object on the production line; a first determination module, used to determine a passable channel, and a travel route and travel parameters of the automatic guided vehicle based on the depth data, position data and preset conditions; a second determination module, used to determine the position and shape of the object based on the depth data and position data; a second control module 34, used to control the automatic guided vehicle to move according to the travel route and travel parameters, and establish a production line model of the production line based on the position and shape of the object.

[0009] An electronic device, the electronic device comprising: a processor, a memory and a communication bus;

[0010] The communication bus is used to realize the communication connection between the processor and the memory;

[0011] The processor is used to execute the digital twin-based production line modeling program in the memory to implement the following steps:

[0012] In the process of controlling the automatic guided vehicle to move on the production line, environmental data around the automatic guided vehicle is collected; the environmental data at least includes: depth data and position data of each object on the production line;

[0013] Determine the navigable channels, the route and parameters of the automated guided vehicle based on the depth data, location data and preset conditions;

[0014] Determine the position and shape of the object based on the depth data and the position data;

[0015] Control the automated guided vehicle to move according to the route and parameters, and build a production line model of the production line based on the position and shape of the object.

[0016] A computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps of the above method.

[0017] The production line modeling method based on digital twin provided in the embodiment of the present application determines the passable channel, the passable route and the passable parameters of the automatic guided vehicle based on the depth data and position data collected in real time by the automatic guided vehicle and the preset conditions, so that the automatic guided vehicle can realize autonomous obstacle avoidance. When encountering complex terrain such as a variety of debris piled on the production line, it can also make real-time and efficient judgments on the passable channel, so as to avoid irregular or even dynamic obstacles, and maximize the passage efficiency, safety and reliability of the automatic guided vehicle during movement, and solve the problem that a large amount of debris or cargo boxes may be placed in the production line during the production process, affecting the freight route of the automatic guided vehicle and being unfavorable to the safe production of the production line. At the same time, in the process of the movement of the automatic guided vehicle, the position and shape of the object are obtained in real time, and the position and shape of each object in the production line model are updated in real time, so as to realize the real-time update and dynamic display of the production line model, and solve the problem that the existing technology can only reflect the static production line, and cannot reflect the movement or replacement of the items in the production line, which greatly improves the real-time performance of the digital twin scene and expands the application scene of the present application to the greatest extent. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 A schematic diagram of the implementation process of a production line modeling method based on digital twins provided in the embodiment of the present application Figure 1 ;

[0019] Figure 2 A schematic diagram of the implementation process of a production line modeling method based on digital twins provided in the embodiment of the present application Figure 2 ;

[0020] Figure 3 A schematic diagram of the composition structure of a production line modeling device based on digital twins provided in an embodiment of the present application;

[0021] Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0022] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0023] It should be understood that the "embodiments of the present application" or "the aforementioned embodiments" mentioned throughout the specification mean that specific features, structures or characteristics related to the embodiments are included in at least one embodiment of the present application. Therefore, "in the embodiments of the present application" or "in the aforementioned embodiments" appearing throughout the specification may not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics may be combined in one or more embodiments in any suitable manner. In the various embodiments of the present application, the size of the sequence numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application. The above-mentioned sequence numbers of the embodiments of the present application are for description only and do not represent the advantages and disadvantages of the embodiments.

[0024] Unless otherwise specified, the electronic device performs any step in the embodiments of the present application, and the processor of the electronic device may perform the step. It is also worth noting that the embodiments of the present application do not limit the order in which the electronic device performs the following steps. In addition, the methods used to process data in different embodiments may be the same method or different methods. It should also be noted that any step in the embodiments of the present application can be independently executed by the electronic device, that is, when the electronic device executes any step in the following embodiments, it may not rely on the execution of other steps.

[0025] It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0026] The embodiment of the present application provides a production line modeling method based on digital twins, which can be applied to electronic devices, such as servers, laptops, tablet computers, desktop computers, smart TVs, set-top boxes, mobile devices, and other devices with data processing capabilities. Figure 1 As shown, the method includes the following steps 101 to 104:

[0027] Step 101: While controlling an automated guided vehicle to move on a production line, collect environmental data around the automated guided vehicle.

[0028] Wherein, the environmental data at least includes: depth data and position data of each object on the production line.

[0029] An Automated Guided Vehicle (AGV) is an unmanned automated device powered by a rechargeable battery and automatically guided. It can accurately reach a designated location and complete a series of tasks according to a predetermined planned path and operational requirements. It is widely used on production lines.

[0030] Environmental data refers to environmental data collected by the automated guided vehicle through its own device. Depth data refers to data obtained through a depth camera. Depth data may include, but is not limited to: depth map, each pixel value in the depth map represents the depth or distance from the corresponding point to the camera, the three-dimensional coordinates of each pixel, and point cloud data (combining depth information and color data to generate point cloud data). Among them, the point cloud is a set of points in three-dimensional space, each point has its corresponding depth and color information. Position data refers to the distance between the automated guided vehicle and each object on the production line.

[0031] In some implementations, step 101 may be specifically implemented by controlling various types of acquisition devices provided on the automatic guided vehicle to acquire environmental data around the automatic guided vehicle in real time.

[0032] Step 102: Determine a navigable passage, a travel route and travel parameters of the automated guided vehicle based on the depth data, the position data and preset conditions.

[0033] The preset conditions are used to reflect the method of determining the traversable channel so that the automated guided vehicle can achieve effective automatic obstacle avoidance.

[0034] In some embodiments, step 102 may be specifically implemented by: determining a navigable channel based on depth data, location data, and preset conditions; and determining a navigable route and traffic parameters based on the navigable channel and the location of the automated guided vehicle.

[0035] Step 103: Determine the position and shape of the object based on the depth data and the position data.

[0036] In some implementations, the depth data and the position data may be integrated based on the position of the automated guided vehicle to obtain the position and shape of the object.

[0037] Step 104: Control the automatic guided vehicle to move according to the travel route and travel parameters, and establish a production line model of the production line based on the position and shape of the object.

[0038] In a feasible implementation, a three-dimensional model of the entire production line can be established in advance to obtain an initial production line model; then, while the automated guided vehicle moves on the production line, the production line model is updated in real time based on the position and shape of the acquired object.

[0039] In another feasible implementation, the depth data and position data of each object on the production line are acquired in real time based on a plurality of automated guided vehicles moving on the production line at the current moment, the position and shape of each object are determined based on the acquired depth data and position data, and a production line model is established in real time based on the acquired positions and shapes of the objects, and the shape and position of each object in the production line model are updated in real time based on a plurality of automated guided vehicles moving on the production line after the current moment.

[0040] The production line modeling method based on digital twin provided in the embodiment of the present application determines the passable channel, the passable route and the passable parameters of the automatic guided vehicle based on the depth data and position data collected in real time by the automatic guided vehicle and the preset conditions, so that the automatic guided vehicle can realize autonomous obstacle avoidance. When encountering complex terrain such as a variety of debris piled on the production line, it can also make real-time and efficient judgments on the passable channel, so as to avoid irregular or even dynamic obstacles, and maximize the passage efficiency, safety and reliability of the automatic guided vehicle during movement, and solve the problem that a large amount of debris or cargo boxes may be placed in the production line during the production process, affecting the freight route of the automatic guided vehicle and being unfavorable to the safe production of the production line. At the same time, in the process of the movement of the automatic guided vehicle, the position and shape of the object are obtained in real time, and the position and shape of each object in the production line model are updated in real time, so as to realize the real-time update and dynamic display of the production line model, and solve the problem that the existing technology can only reflect the static production line, and cannot reflect the movement or replacement of the items in the production line, which greatly improves the real-time performance of the digital twin scene and expands the application scene of the present application to the greatest extent.

[0041] Based on the above embodiments, the present application provides a production line modeling method based on digital twins. Figure 2 As shown, the method comprises the following steps:

[0042] Step 201: While controlling an automated guided vehicle to move on a production line, collect environmental data around the automated guided vehicle.

[0043] Wherein, the environmental data at least includes: depth data and position data of each object on the production line.

[0044] In some embodiments, the specific implementation method of step 201 may be: controlling the automatic guided vehicle to obtain the depth data of each object on the production line within the field of view of the depth camera through its own depth camera; controlling the automatic guided vehicle to collect the position data of each object on the production line through its own positioning device; controlling the automatic guided vehicle to collect the distance data of each object on the production line through its own laser radar; controlling the automatic guided vehicle to collect the inclination angle of the automatic guided vehicle through its own angle determination module.

[0045] In a feasible implementation, multiple depth cameras can be set on the automatic guided vehicle, and these multiple depth cameras are respectively set on the front, rear, left and right sides of the automatic guided vehicle to collect depth data of the front, rear, left and right sides of the automatic guided vehicle.

[0046] The depth data and position data collected by the automated guided vehicle can be used to calculate in real time the position and shape of objects such as the automated guided vehicle, debris on the production line, cargo boxes, production lines, walls, etc., so as to update the position and shape of each object in the production line model in real time.

[0047] Step 202: Determine a first target search area based on the depth data, the position data and a first preset condition.

[0048] Wherein, the first preset condition is a method for determining the first target search area;

[0049] The first target search area refers to all search areas constructed based on the depth data, the position data and the first preset condition. The first preset condition may refer to the construction logic of the search area; for example, the first preset condition may be to construct the search area based on multiple coordinates in the same plane.

[0050] In some implementations, step 202 may be implemented by steps 2021 to 2023 as follows:

[0051] Step 2021: Taking the position of the automatic guided vehicle as a reference, determine from the depth image a plurality of coordinate points whose distances satisfy a first distance threshold, and obtain a first array.

[0052] The depth data at least includes: the depth image, a plurality of coordinate points corresponding to the depth image, and the distance between the coordinate points and the automatic guided vehicle; and the position data includes the position of the automatic guided vehicle.

[0053] Each coordinate point in the depth image may represent one or more pixel points. The first distance threshold may be determined based on the distance between the coordinate point in the depth image and the automatic guided vehicle and a preset distance. For example, the first distance threshold may be a distance interval. If the distance d1 between a coordinate point in the depth image and the automatic guided vehicle is 10 cm and the preset distance dδ is 1 cm, the first distance threshold may be 9 to 11 cm.

[0054] For the depth image, the coordinate points in the image whose distances are within the range of d1±dδ are counted, and the coordinates are stored in the first array Ad1.

[0055] Step 2022: Determine a first search area based on the maximum and minimum values ​​of the horizontal coordinates and the maximum and minimum values ​​of the vertical coordinates in the first array.

[0056] Step 2023: Continue to determine from the depth image multiple coordinate points whose distances satisfy a second distance threshold to obtain a second array, and determine a second search area based on the maximum and minimum values ​​of the horizontal coordinates and the maximum and minimum values ​​of the vertical coordinates in the second array, and repeat this cycle until all coordinate points in the depth data are traversed, and use the obtained search area as the first target search area.

[0057] The second distance threshold may be greater than the first distance threshold or less than the first distance threshold. If the second distance threshold is greater than the first distance threshold, the distance between the second search area and the automatic guided vehicle is greater than the distance between the first search area and the automatic guided vehicle, that is, the first search area is closer to the automatic guided vehicle, and the second search area is farther from the automatic guided vehicle. Similarly, if the second distance threshold is less than the first distance threshold, the distance between the second search area and the automatic guided vehicle is less than the distance between the first search area and the automatic guided vehicle, that is, the second search area is closer to the automatic guided vehicle, and the first search area is farther from the automatic guided vehicle.

[0058] Step 203: Determine a passable channel based on the distance difference between every two adjacent first target search areas and a second preset condition.

[0059] Among them, the second preset condition is the method of determining the passable channel.

[0060] The second preset condition may be a condition set based on the distance difference between adjacent search areas, or may be a condition set based on the distance difference between adjacent search areas and the size of the search area itself.

[0061] In some implementations, step 203 may be implemented by steps 2031 to 2033 as follows:

[0062] Step 2031: Determine the distance difference between every two adjacent first target search areas.

[0063] Step 2032: If the largest distance difference is greater than or equal to a first preset threshold, determine a first target search area with a larger distance from the two first target search areas that produce the largest distance difference, and obtain a second target search area.

[0064] The first preset threshold can be set to 1.5m. Since the difference between the distance between the passable channel and the automatic guided vehicle and the distance between the obstacle and the automatic guided vehicle is generally greater than 1.5m, therefore, setting the first preset threshold to 1.5m can better analyze the passable channel, simplify the extremely complex channel judgment problem through clever design, and do not require complex algorithm modeling, which greatly improves the intelligence level of this application, realizes concise, efficient, and highly intelligent production line modeling based on digital twins, and can perform real-time and efficient channel judgment when encountering complex terrain such as obstacles, realizing the ability to avoid irregular or even dynamic obstacles, and expanding the application scenarios of this application to the greatest extent.

[0065] Step 2033: If the width of the second target search area is greater than or equal to a second preset threshold, and the height of the second target search area is greater than or equal to a third preset threshold, determine that the second target search area is a passable passage.

[0066] The second preset threshold and the third preset threshold can be preset. Specifically, the second preset threshold can be set according to the width of the automatic guided vehicle; for example, the second preset threshold is greater than the width of the automatic guided vehicle. Specifically, the third preset threshold can be set according to the height of the automatic guided vehicle; for example, the third preset threshold is greater than the height of the automatic guided vehicle. In this way, it is ensured that the automatic guided vehicle can pass through the passable channel.

[0067] Step 204: Determine the travel route and the travel parameters based on the navigable passage.

[0068] Here, step 202 to step 204 correspond to the aforementioned step 102, and the specific implementation of the aforementioned step 102 may be referred to during implementation.

[0069] The travel parameters may include displacement distance, rotation angle, travel speed, driving force, etc.

[0070] In some implementations, step 204 may be implemented by steps 2041 to 2043 as follows:

[0071] Step 2041: determine the center point of the navigable passage on the horizon, the angle between the center point and the driving direction of the automatic guided vehicle, and the first distance between the center point and the position of the automatic guided vehicle.

[0072] The angle between the center point and the driving direction of the automatic guided vehicle can also be called the tilt angle. The first distance can be the straight-line distance between the center point and the position of the automatic guided vehicle.

[0073] Step 2042: Based on the angle and the first distance, determine a travel route and a first travel speed for the automatic guided vehicle to move to the center point.

[0074] The passing route may be to first drive to the same straight line as the center point and then move forward; the passing route may also be to drive directly to the center point along the angle. The first passing speed may be 0.4 m per second.

[0075] Step 2043: Determine the speed at which the automatic guided vehicle moves in the navigable channel based on the distance between the automatic guided vehicle and the objects on the left and right sides and a fourth preset threshold; the passage parameters include the first passage speed and the speed at which the automatic guided vehicle moves in the navigable channel.

[0076] The fourth preset threshold is used to determine whether the automatic guided vehicle is within a safe range.

[0077] In a feasible implementation, the position DL of the vehicle relative to the object on the left and the position DR of the vehicle relative to the object on the right can be obtained by using the values ​​of the laser radar at 90 degrees and 270 degrees set on the automatic guided vehicle. The 125° of the laser radar corresponds to the 0° of the depth camera's viewing angle.

[0078] In some embodiments, the specific implementation method of step 2043 may be: if the area of ​​the traversable passage satisfies the first sub-preset threshold, and the distance between the automatic guided vehicle and the objects on the left and right sides satisfies the second sub-preset threshold, then the speed at which the automatic guided vehicle moves in the traversable passage is determined to be the second passing speed; if the area of ​​the traversable passage does not satisfy the first sub-preset threshold, and / or the distance between the automatic guided vehicle and the objects on the left and right sides does not satisfy the second sub-preset threshold, then the speed at which the automatic guided vehicle moves in the traversable passage is determined to be the third passing speed; the second passing speed is higher than the third passing speed.

[0079] The first sub-preset threshold can be an interval consisting of two thresholds; one of the thresholds is 1.2 times the product of the second preset threshold and the third preset threshold, and the other is the product of the second preset threshold and the third preset threshold. It is used to achieve that when the area of ​​the passable channel is large enough for the AGV intelligent transport robot to pass but is still small, the moving speed of the AGV intelligent transport robot is reduced to prevent the AGV intelligent transport robot from moving too fast and causing the collapse of the passable channel, thereby maximizing the application scenarios and reliability of the present application.

[0080] The second sub-preset threshold can be determined according to the width of the automated guided vehicle. When DL and DR are both greater than or equal to the second sub-preset threshold, it proves that DL and DR are large enough. The automated guided vehicle resumes the second speed at this time, and it will not cause the collapse of the passable passage caused by the excessive moving speed. It can also be judged as having passed the passable passage. The AGV intelligent transport robot is cleverly implemented to autonomously judge whether to pass through the passage. If it passes through the passage, the speed of the AGV intelligent transport robot is restored to the original second speed, which further simplifies the complex problem and further improves the intelligence and usability of the present application.

[0081] Step 205: Determine the position and shape of the object based on the depth data and the position data.

[0082] In some embodiments, step 205 may be specifically implemented by integrating the depth data and position data of each pixel point around the automatic guided vehicle based on the position of the automatic guided vehicle to obtain the shape and position of objects around the automatic guided vehicle.

[0083] In a feasible implementation, based on the position of the automatic guided vehicle, the coordinate points of multiple pixel points representing the same object can be integrated to obtain the shape of the object; then, the center point of the object on the horizon is used as the coordinate of the object.

[0084] Step 206: Control the automatic guided vehicle to move according to the travel route and travel parameters, and establish a production line model of the production line based on the position and shape of the object.

[0085] Based on the above embodiment, the embodiment of the present application provides a production line modeling method based on digital twins, which also includes the following steps 207 to 208:

[0086] Step 207: If the tilt angle of the automatic guided vehicle does not meet the angle threshold, adjust the posture and speed of the automatic guided vehicle based on the tilt angle.

[0087] Among them, the environmental data includes the inclination angle, which includes the climbing angle of the automatic guided vehicle, the angle of deviation between the forward direction of the automatic guided vehicle and the center point of the navigable passage on the horizon; the passage parameters include the posture and speed of the automatic guided vehicle.

[0088] Angle threshold refers to the preset safety angle value.

[0089] In actual scenarios, if the AGV is climbing a slope, it needs to reduce its speed to increase its climbing power. If the AGV's driving direction deviates, it needs to adjust its driving direction in time to ensure the AGV's safe passage.

[0090] Step 208: Based on the adjusted posture and speed of the automatic guided vehicle, control the automatic guided vehicle to move to a position where the tilt angle meets the angle threshold.

[0091] If the tilt angle of the automatic guided vehicle is greater than or equal to the preset safety angle value, the forward posture of the tilt angle of the automatic guided vehicle is improved, and the automatic guided vehicle moves forward at a low speed until the tilt angle value of the tilt angle of the automatic guided vehicle is less than the preset safety angle.

[0092] It should be noted that, for the description of the same steps and the same contents in this embodiment as those in other embodiments, reference can be made to the description in other embodiments and will not be repeated here.

[0093] The production line modeling method based on digital twin provided in the embodiment of the present application determines the passable channel, the passable route and the passable parameters of the automatic guided vehicle based on the depth data and position data collected in real time by the automatic guided vehicle and the preset conditions, so that the automatic guided vehicle can realize autonomous obstacle avoidance. When encountering complex terrain such as a variety of debris piled on the production line, it can also make real-time and efficient judgments on the passable channel, so as to avoid irregular or even dynamic obstacles, and maximize the passage efficiency, safety and reliability of the automatic guided vehicle during movement, and solve the problem that a large amount of debris or cargo boxes may be placed in the production line during the production process, affecting the freight route of the automatic guided vehicle and being unfavorable to the safe production of the production line. At the same time, in the process of the movement of the automatic guided vehicle, the position and shape of the object are obtained in real time, and the position and shape of each object in the production line model are updated in real time, so as to realize the real-time update and dynamic display of the production line model, and solve the problem that the existing technology can only reflect the static production line, and cannot reflect the movement or replacement of the items in the production line, which greatly improves the real-time performance of the digital twin scene and expands the application scene of the present application to the greatest extent.

[0094] The following describes the application of the digital twin-based production line modeling method provided in an embodiment of the present application in actual scenarios.

[0095] The data twin model can monitor the status of the production line in real time and provide the latest operating information, thereby helping operators to understand the operation status of the production line or equipment in a timely manner.

[0096] The existing production line modeling method generally uses 3D modeling technology to construct the data representation of objects in 3D space (3D model). The 3D model can be represented by a point cloud composed of the 3D coordinate points of the object. The point cloud can be connected using triangular meshes, lines, and polygonal meshes to reconstruct the surface of the model. 3D models can be used in movies, games, manufacturing and other fields. 3D modeling technology is a multidisciplinary research field and an important application of computer graphics and image processing in engineering.

[0097] The existing 3D modeling solutions can only reflect the static 3D scene of the production line in real time, but cannot realize the dynamic 3D scene of the production line. For example, in a flexible production line, the types and locations of production equipment and handling equipment used in production often change. In addition, after the static 3D scene is built, there is a lack of effective means to update the static 3D scene in a timely manner. For example, during the production process, a large amount of debris or cargo boxes may be placed on the production line, which may even affect the freight route of the AGV robot, which is not conducive to the safe production of the production line. At the same time, the existing 3D modeling solutions cannot display the location of the debris or cargo boxes on the AGV robot and the production line in real time, and the real-time performance of the 3D scene is poor and the function is single.

[0098] Therefore, the prior art needs to be further developed.

[0099] The present application provides a production line modeling method based on digital twins, comprising:

[0100] S100, control each AGV robot to move at a second passing speed after entering the production line, use the depth camera set on the AGV robot to collect environmental data around the AGV robot, and use the depth data and position data collected by the depth camera set on the AGV robot and the positioning device to establish a three-dimensional model of the production line.

[0101] In some embodiments, a three-dimensional model of the production line is established using depth data and position data collected by a depth camera and a positioning device provided on the AGV robot, including: using the positioning device on the AGV robot to obtain the current positioning data of the AGV robot, using the depth camera provided on the AGV robot to obtain the depth data of each pixel within the current field of view of the camera, integrating the obtained current positioning data of the AGV robot and the depth data of each pixel within the current field of view of the camera to obtain the three-dimensional data within the current field of view of the camera, and establishing a three-dimensional model of the production line based on the three-dimensional data within the current field of view of the camera.

[0102] It should be noted that in the technical solution of the present application, an initial three-dimensional model of the production line can be established with the help of production line drawings, and an AGV robot virtual prototype and virtual simulation platform can be established. After the switch receives the data sent by the relay machine, it forwards it to the host computer. After the host computer solves the data, it obtains the operating status of the AGV robot and the production line environment information, and sets the initial starting point of the AGV robot.

[0103] In some embodiments, before step S100, the process includes: pre-setting a first passing speed, a second passing speed, a third passing speed, a first preset threshold, a second preset threshold, a third preset threshold, a fourth preset threshold, a first sub-preset threshold, a second sub-preset threshold, and a preset safety angle.

[0104] It can be understood that the first passing speed, the second passing speed, the third passing speed, the first preset threshold, the second preset threshold, the third preset threshold, the fourth preset threshold (the first sub-preset threshold and the second sub-preset threshold), and the preset safety angle can be set according to the actual needs of the users of this application. This application does not limit the specific values ​​of the above parameters, as long as it is applicable to the production line modeling method based on digital twins proposed in this application.

[0105] For example, the second passing speed can be 1 meter per second, the first passing speed and the third passing speed can be 0.4 meters per second, the second passing speed is greater than the third passing speed, so as to realize the hierarchical control of the moving speed of the automatic guided vehicle, the second preset threshold is greater than the width of the AGV robot, the third preset threshold is greater than the height of the AGV robot, so as to ensure that the AGV robot can pass through the passable passage, one value in the first sub-preset threshold is 1.2 times the product of the second preset threshold and the third preset threshold, and another value in the first sub-preset threshold is the product of the second preset threshold and the third preset threshold, so as to realize that when the area of ​​the passable passage is small enough for the AGV robot to pass through, the moving speed of the AGV robot is reduced to prevent the AGV robot from moving too fast and causing the passable passage to collapse, thereby maximizing the application scenario and reliability of the present application. Preferably, the first preset threshold is 1.5 meters. Setting the first preset threshold to 1.5 meters is the result of a large number of experiments conducted by the technical personnel of this application. Since the difference between the distance from the end of the passable channel to the AGV robot and the distance from the obstacle to the AGV robot is usually greater than 1.5 meters, setting the first preset threshold to 1.5 meters can be a good way to analyze whether a passable channel is found. The extremely complex channel judgment problem is simplified through clever design, and no complex algorithm modeling is required. The intelligence level of this application is greatly improved, and simple, efficient, and highly intelligent production line modeling based on digital twins is realized. When encountering complex terrain such as obstacles, real-time and efficient channel judgment can be performed, which enables the ability to avoid irregular and even dynamic obstacles, and expands the application scenarios of this application to the greatest extent.

[0106] S200, in the process of each AGV robot moving according to the preset handling route, the collected data is processed, and the passable channel is judged according to the preset judgment rules using the processed depth camera data, and the travel route (the above-mentioned passable route) and travel rules (the above-mentioned pass parameters) are formulated based on the judgment results. When the AGV robot moves according to the formulated travel route and travel rules, the distance data between the AGV robot and the channel is collected in real time using the laser radar set on the AGV robot, and the inclination angle data of the AGV robot is obtained by using the angle determination module set on the AGV robot. The passability of the AGV robot in the channel is judged in real time through the distance data, and the posture of the AGV robot is judged whether it is in a safe range through the obtained inclination angle data. The angle determination module can be a strapdown inertial navigation module.

[0107] In some embodiments, processing the collected data includes: first obtaining a set of depth arrays {Ad1, Ad2, ..., Adn} and corresponding angles and distances {d1, d2, ..., dn} in the depth image data. Among them, 125° of the laser radar corresponds to 0° of the depth camera's viewing angle, counting the points in the image within the range of d1±dσ, storing the coordinates in the array Ad1, and obtaining the maximum and minimum values ​​of the horizontal coordinates and the maximum and minimum values ​​of the vertical coordinates in the array Ad1, establishing a target search frame M1 based on the maximum and minimum values ​​of the horizontal coordinates and the maximum and minimum values ​​of the vertical coordinates in Ad1, counting the points in the image within the range of d2±dσ, storing the coordinates in the array Ad2, ..., and so on, so that the depth image is divided into a series of target search frames Mn.

[0108] In some embodiments, the processed depth camera data is used to determine the passable channel according to preset judgment rules, and the travel route and travel rules are formulated based on the judgment results, including: respectively calculating the distance difference of adjacent target search boxes, and determining whether the maximum distance difference is greater than or equal to a first preset threshold value. If so, determining whether the width of the target search box with a larger distance between the two target search boxes that produce the maximum distance difference is greater than or equal to a second preset threshold value, and at the same time determining whether the height of the target search box is greater than or equal to a third preset threshold value. If so, the area indicated by the target search box is a passable channel, then the deviation angle between the center of the target search box and the driving direction of the AGV robot is calculated, and the distance that the AGV robot needs to move left or right is calculated based on the distance and the deviation angle. The AGV robot moves the distance left or right so that the forward direction of the AGV robot and the center of the target search box are located in the same vertical plane. Then, the AGV robot enters the passable channel along the intersection of the vertical plane and the horizon.

[0109] In some embodiments, the passability of the AGV robot in the channel is judged in real time through distance data, including: the 0 degree of the laser radar always points to the opposite direction of the AGV robot's advance, and the position DL relative to the object on the left and the position DR relative to the object on the right are obtained through the values ​​of 90 degrees and 270 degrees of the laser radar. When the area of ​​the target search box is less than or equal to the first sub-preset threshold and greater than or equal to the second sub-preset threshold, the AGV robot enters the channel and moves at the third passing speed, moves at the third passing speed, and judges whether the changes in DL and DR are both greater than or equal to the second sub-preset threshold. If both are greater than the second sub-preset threshold, the AGV robot moves at the second passing speed.

[0110] In some embodiments, the obtained inclination angle data is used to determine whether the posture of the AGV robot is within a safe range, including: if the inclination angle of the AGV robot is greater than or equal to a preset safety angle value, the AGV robot improves its forward posture and moves forward at a first passing speed until the inclination angle of the AGV robot is less than the preset safety angle, and determines whether it has reached the end point of the preset transport route. If it has reached the end point of the preset transport route, navigation ends.

[0111] It should be noted that the 0 degree of the laser radar always points in the opposite direction of the AGV robot's advance. Through the values ​​of 90 degrees and 270 degrees of the laser radar, when the AGV robot enters the channel, it obtains its own position DL relative to the object on the left and its position DR relative to the object on the right. When the area of ​​the target search box is less than or equal to the first sub-preset threshold and greater than or equal to the second sub-preset threshold, the AGV robot moves at the third passing speed and determines whether the changes in DL and DR are both greater than or equal to the second sub-preset threshold. If both are greater than the second sub-preset threshold, the AGV robot moves at the second passing speed.

[0112] It is understandable that when the changes in DL and DR are greater than or equal to the second sub-preset threshold, it proves that DL and DR are large enough, and the AGV robot's restoration of the second passage speed will not cause the passage to collapse due to excessive movement speed, and it can also be judged that the passage has been passed. The AGV robot can cleverly determine whether to pass through the passage autonomously. If it passes through the passage, the speed of the AGV robot is restored to the original second passage speed, which further simplifies complex problems and further improves the intelligence and usability of this application.

[0113] S300, using the depth data and position data collected by the depth camera and positioning device installed on each AGV robot, calculate the position and model of each AGV robot, debris or cargo box on the production line in real time, and update the three-dimensional model of the production line and each AGV robot in real time.

[0114] Specifically, a plurality of depth cameras are provided, and the plurality of depth cameras are respectively provided on the front side, the rear side, the left side and the right side of the AGV robot, for collecting depth images of the front side, the rear side, the left side and the right side of the AGV robot.

[0115] In some embodiments, the depth data and position data collected by the depth camera and positioning device installed on each AGV robot are used to calculate the position and model of each AGV robot, the debris or cargo box on the production line in real time, and the three-dimensional model of the production line and each AGV robot is updated in real time, including: using the positioning device installed on the AGV robot to obtain the current positioning data of the AGV robot, using the front, rear, left and right depth cameras installed on the AGV robot to obtain the depth data of each pixel point within the current field of view of the front, rear, left and right cameras, integrating the obtained current positioning data of the AGV robot with the depth data of each pixel point within the current field of view of the front, rear, left and right cameras to obtain the three-dimensional data within the current field of view of the camera, and updating the three-dimensional model of the production line and each AGV robot in real time according to the three-dimensional data within the current field of view of the front, rear, left and right cameras.

[0116] It is understandable that this application sets an effective automatic obstacle avoidance scheme for the AGV robot, and can make real-time and efficient judgments on passable channels when encountering complex terrain such as debris on the production line, so as to avoid irregular or even dynamic obstacles, and maximize the efficiency, safety and reliability of the transportation process of the AGV freight robot. It solves the problem that a large amount of debris or cargo boxes may be placed in the production line during the production process, which may even affect the freight route of the AGV robot and be detrimental to the safe production of the production line. At the same time, this application can not only realize the static 3D scene of the production line in real time, but also use the depth camera set on each AGV freight robot to realize the real-time update and dynamic display of the dynamic 3D scene of the production line, solving the problem in the prior art that when the static 3D scene is built, there is a lack of effective means to update the static 3D scene in time. This application realizes the real-time display of the position and model of each AGV robot, debris or cargo box on the production line in the digital twin model of the production line, greatly improving the real-time performance of the 3D digital twin scene, and expanding the application scenario of this application to the greatest extent.

[0117] The present application provides another embodiment, which provides a production line modeling system based on digital twins, comprising:

[0118] An acquisition module, including a depth camera provided on the AGV robot, is used to collect surrounding image data of the AGV robot;

[0119] The control module is used to control each AGV robot to move at a second passing speed after entering the production line, and to establish a three-dimensional model of the production line using the depth data and position data collected by the depth camera and the positioning device set on the AGV robot; it is used to process the collected data in the process of each AGV robot moving according to the preset handling route, and to use the processed depth camera data to judge the passable channel according to the preset judgment rule, and to formulate the travel route and travel rules based on the judgment result. When the AGV robot moves according to the formulated travel route and travel rules, the laser radar set on the AGV robot is used to collect the distance data between the AGV robot and the channel in real time, and the angle determination module set on the AGV robot is used to obtain the inclination angle data of the AGV robot, and the passability of the AGV robot in the channel is judged in real time through the distance data, and the posture of the AGV robot is judged whether it is in a safe range through the obtained inclination angle data; the angle determination module can be a strapdown inertial navigation module; it is used to use the depth data and position data collected by the depth camera and the positioning device set on each AGV robot to calculate the position and model of each AGV robot, the sundries or cargo boxes on the production line in real time, and update the three-dimensional model of the production line and each AGV robot in real time.

[0120] Based on the foregoing embodiments, the embodiments of the present application provide a production line modeling device based on digital twins, which includes the modules included and the units included in the modules, which can be implemented by a processor in a control device; of course, it can also be implemented by a specific logic circuit; in the implementation process, the processor can be a central processing unit (CPU), a microprocessor (MPU), a digital signal processor (DSP) or a field programmable gate array (FPGA), etc.

[0121] The production line modeling device based on digital twin, such as Figure 3 As shown, including:

[0122] The first control module 31 is used to control the automatic guided vehicle to collect environmental data around the automatic guided vehicle during its movement on the production line; the environmental data at least includes: depth data and position data of each object on the production line;

[0123] A first determination module 32, configured to determine a navigable passage, a navigable route and navigable parameters of the automated guided vehicle based on the depth data, the position data and preset conditions;

[0124] A second determination module 33, configured to determine the position and shape of the object based on the depth data and the position data;

[0125] The second control module 34 is used to control the automatic guided vehicle to move according to the travel route and travel parameters, and to establish a production line model of the production line based on the position and shape of the object.

[0126] In some embodiments, the first determination module 32 includes:

[0127] A first determining unit, configured to determine a first target search area based on the depth data, the position data and a first preset condition; the first preset condition is a method for determining the first target search area;

[0128] a second determining unit, configured to determine a passable channel based on a distance difference between every two adjacent first target search areas and a second preset condition; wherein the second preset condition is a method for determining the passable channel;

[0129] The third determination unit is used to determine the passable route and the pass parameter based on the passable channel; the preset conditions include the first preset conditions and the second preset conditions.

[0130] In some embodiments, the first determining unit is configured to:

[0131] Taking the position of the automatic guided vehicle as a reference, a plurality of coordinate points whose distances satisfy a first distance threshold are determined from the depth image to obtain a first array; the depth data at least includes: the depth image, a plurality of coordinate points corresponding to the depth image, and the distances between the coordinate points and the automatic guided vehicle; the position data includes the position of the automatic guided vehicle;

[0132] Determine a first search area based on the maximum and minimum values ​​of the abscissa and the maximum and minimum values ​​of the ordinate in the first array;

[0133] Continue to determine from the depth image multiple coordinate points whose distances satisfy a second distance threshold to obtain a second array, and determine a second search area based on the maximum and minimum values ​​of the horizontal coordinates and the maximum and minimum values ​​of the vertical coordinates in the second array, and repeat this cycle until all coordinate points in the depth data are traversed, and use the obtained search area as the first target search area.

[0134] In some embodiments, the second determining unit is configured to:

[0135] Determine the distance difference between every two adjacent first target search areas;

[0136] If the largest distance difference is greater than or equal to a first preset threshold, determining a first target search area with a larger distance from the two first target search areas that produce the largest distance difference, to obtain a second target search area;

[0137] If the width of the second target search area is greater than or equal to a second preset threshold, and the height of the second target search area is greater than or equal to a third preset threshold, it is determined that the second target search area is a passable passage.

[0138] In some embodiments, the third determining unit is configured to:

[0139] A first determining subunit is used to determine the center point of the navigable passage on the horizon, the angle between the center point and the driving direction of the automatic guided vehicle, and the first distance between the center point and the position of the automatic guided vehicle;

[0140] A second determining subunit is used to determine a travel route and a first travel speed of the automatic guided vehicle moving to the center point based on the angle and the first distance;

[0141] The third determination subunit is used to determine the speed at which the automatic guided vehicle moves in the passable channel based on the distance between the automatic guided vehicle and the objects on the left and right sides and a fourth preset threshold; the passage parameters include the first passage speed and the speed at which the automatic guided vehicle moves in the passable channel.

[0142] In some embodiments, the third determining subunit is configured to:

[0143] If the area of ​​the passable passage meets the first sub-preset threshold value, and the distance between the automatic guided vehicle and the objects on the left and right sides meets the second sub-preset threshold value, then determining that the speed of the automatic guided vehicle moving in the passable passage is the second pass speed;

[0144] If the area of ​​the traversable passage does not satisfy the first sub-preset threshold, and / or the distance between the automatic guided vehicle and the objects on the left and right sides does not satisfy the second sub-preset threshold, the speed at which the automatic guided vehicle moves in the traversable passage is determined to be a third passing speed; and the second passing speed is higher than the third passing speed.

[0145] In some embodiments, the apparatus further comprises:

[0146] an adjustment unit, configured to adjust the posture and speed of the automatic guided vehicle based on the tilt angle if the tilt angle of the automatic guided vehicle does not meet an angle threshold; the environmental data includes the tilt angle, the tilt angle includes a climbing angle of the automatic guided vehicle, an angle of deviation between a forward direction of the automatic guided vehicle and a center point of the navigable passage on the horizon; the traffic parameters include the posture and speed of the automatic guided vehicle;

[0147] A third control unit is configured to control the automatic guided vehicle to move to a position where the tilt angle satisfies the angle threshold based on the adjusted posture and speed of the automatic guided vehicle.

[0148] In some embodiments, the first control module is used to control the automatic guided vehicle to obtain the depth data of each object on the production line within the field of view of the depth camera through its own depth camera; control the automatic guided vehicle to collect the position data of each object on the production line through its own positioning device; control the automatic guided vehicle to collect the distance data of each object on the production line through its own laser radar; control the automatic guided vehicle to collect the inclination angle of the automatic guided vehicle through its own angle determination module.

[0149] In some embodiments, depth cameras are arranged around the automatic guided vehicle, and the depth cameras are used to collect depth data of each pixel point within the field of view;

[0150] The second determination module 33 is used to integrate the depth data and position data of each pixel point around the automatic guided vehicle based on the position of the automatic guided vehicle to obtain the shape and position of objects around the automatic guided vehicle.

[0151] The description of the above device embodiment is similar to the description of the above method embodiment, and has similar beneficial effects as the method embodiment. In some embodiments, the functions or modules included in the device provided in the embodiment of the present application can be used to execute the method described in the above method embodiment. For technical details not disclosed in the device embodiment of the present application, please refer to the description of the method embodiment of the present application for understanding.

[0152] It should be noted that in the embodiment of the present application, if the above-mentioned data processing method is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the relevant technology can be embodied in the form of a software product, which is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods of each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a U disk, a mobile hard disk, a read-only memory (ROM), a disk or an optical disk. In this way, the embodiment of the present application is not limited to any specific hardware, software or firmware, or any combination of hardware, software, and firmware.

[0153] An embodiment of the present application provides a computer device, including a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the processor executes the program, some or all of the steps in the above method are implemented.

[0154] The embodiment of the present application provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, some or all of the steps in the above method are implemented. The computer-readable storage medium can be transient or non-transient.

[0155] An embodiment of the present application provides a computer program, including a computer-readable code. When the computer-readable code is run in a computer device, a processor in the computer device executes some or all of the steps for implementing the above method.

[0156] The embodiment of the present application provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and when the computer program is read and executed by a computer, some or all of the steps in the above method are implemented. The computer program product can be implemented specifically by hardware, software or a combination thereof. In some embodiments, the computer program product is specifically embodied as a computer storage medium, and in other embodiments, the computer program product is specifically embodied as a software product, such as a software development kit (SDK) and the like.

[0157] It should be noted here that the description of the various embodiments above tends to emphasize the differences between the various embodiments, and the same or similar aspects can be referenced to each other. The description of the above device, storage medium, computer program and computer program product embodiments is similar to the description of the above method embodiment, and has similar beneficial effects as the method embodiment. For technical details not disclosed in the embodiments of the device, storage medium, computer program and computer program product of this application, please refer to the description of the method embodiment of this application for understanding.

[0158] Figure 4 is a schematic diagram of a hardware entity of an electronic device in an embodiment of the present application, such as Figure 4 As shown, the hardware entity of the electronic device 400 includes: a processor 401, a communication interface 402 and a memory 403, wherein:

[0159] The processor 401 generally controls the overall operation of the electronic device 400 , and the overall operation may be to implement the processing method provided in the embodiment of the present application.

[0160] The communication interface 402 enables the computer device to communicate with other terminals or servers through a network.

[0161] The memory 403 is configured to store instructions and applications executable by the processor 401, and can also cache data to be processed or processed by the processor 401 and each module in the electronic device 400 (for example, image data, audio data, voice communication data, and video communication data), which can be implemented by flash memory (FLASH) or random access memory (Random Access Memory, RAM). Data transmission can be performed between the processor 401, the communication interface 402, and the memory 403 through the bus 404.

[0162] An embodiment of the present application provides a computer storage medium, which stores one or more programs. The one or more programs can be executed by one or more processors to implement the steps of the processing method of any of the above embodiments.

[0163] It should be noted here that the description of the above storage medium and device embodiments is similar to the description of the above method embodiments, and has similar beneficial effects as the method embodiments. For technical details not disclosed in the storage medium and device embodiments of this application, please refer to the description of the method embodiments of this application for understanding.

[0164] The processor may be at least one of an Application Specific Integrated Circuit (ASIC), a Digital Signal Processor (DSP), a Digital Signal Processing Device (DSPD), a Programmable Logic Device (PLD), a Field Programmable Gate Array (FPGA), a Central Processing Unit (CPU), a controller, a microcontroller, and a microprocessor. It is understandable that the electronic device that implements the functions of the processor may also be other, and the embodiments of the present application are not specifically limited.

[0165] The above-mentioned computer storage medium / memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a flash memory (Flash Memory), a magnetic surface memory, an optical disk, or a compact disc read-only memory (CD-ROM) and the like; it can also be various terminals including one or any combination of the above-mentioned memories, such as mobile phones, computers, tablet devices, personal digital assistants, etc.

[0166] It should be understood that "one embodiment" or "an embodiment" mentioned throughout the specification means that specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. It should be understood that in various embodiments of the present application, the size of the serial number of each step / process mentioned above does not mean the order of execution, and the execution order of each step / process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present application. The serial numbers of the embodiments of the present application mentioned above are for description only and do not represent the advantages and disadvantages of the embodiments.

[0167] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0168] The above are only implementation methods of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application.

Claims

1. A production line modeling method based on digital twins, the method comprising: In the process of controlling the automatic guided vehicle to move on the production line, collecting environmental data around the automatic guided vehicle; The environmental data at least includes: depth data and position data of each object on the production line; Determine a navigable passage, a navigable route and navigable parameters of the automated guided vehicle based on the depth data, the position data and preset conditions; Determining the position and shape of the object based on the depth data and the position data; The automatic guided vehicle is controlled to move according to the travel route and travel parameters, and a production line model of the production line is established based on the position and shape of the object.

2. According to the production line modeling method based on digital twins in claim 1, the determining of the navigable channel, and the passage route and passage parameters of the automatic guided vehicle based on the depth data, the position data and the preset conditions comprises: Determining a first target search area based on the depth data, the position data and a first preset condition; The first preset condition is a method for determining the first target search area; Determining a passable channel based on a distance difference between every two adjacent first target search areas and a second preset condition; The second preset condition is a method for determining the passable passage; Based on the passable passage, the pass route and the pass parameter are determined; the preset conditions include the first preset conditions and the second preset conditions.

3. According to the production line modeling method based on digital twins in claim 2, the determining of the first target search area based on the depth data, the position data and the first preset condition comprises: Taking the position of the automatic guided vehicle as a reference, determining from the depth image a plurality of coordinate points whose distances satisfy a first distance threshold, to obtain a first array; The depth data includes at least: the depth image, a plurality of coordinate points corresponding to the depth image, and the distance between the coordinate points and the automatic guided vehicle; the position data includes the position of the automatic guided vehicle; Determine a first search area based on the maximum and minimum values ​​of the abscissa and the maximum and minimum values ​​of the ordinate in the first array; Continue to determine from the depth image multiple coordinate points whose distances satisfy a second distance threshold to obtain a second array, and determine a second search area based on the maximum and minimum values ​​of the horizontal coordinates and the maximum and minimum values ​​of the vertical coordinates in the second array, and repeat this cycle until all coordinate points in the depth data are traversed, and use the obtained search area as the first target search area.

4. The production line modeling method based on digital twin according to claim 2 or 3, wherein determining the passable channel based on the distance difference between each two adjacent first target search areas and the second preset condition comprises: Determine the distance difference between every two adjacent first target search areas; If the largest distance difference is greater than or equal to a first preset threshold, determining a first target search area with a larger distance from the two first target search areas that produce the largest distance difference, to obtain a second target search area; If the width of the second target search area is greater than or equal to a second preset threshold, and the height of the second target search area is greater than or equal to a third preset threshold, it is determined that the second target search area is a passable passage.

5. According to the production line modeling method based on digital twins according to claim 2 or 3, the determining of the passable route and the passable parameters based on the passable channel comprises: Determine a center point of the navigable passage on the horizon, an angle between the center point and the travel direction of the automatic guided vehicle, and a first distance between the center point and a position of the automatic guided vehicle; Based on the angle and the first distance, determining a travel route and a first travel speed of the automatic guided vehicle moving to the center point; Based on the distance between the automatic guided vehicle and the objects on the left and right sides and a fourth preset threshold, the speed at which the automatic guided vehicle moves in the navigable channel is determined; the passage parameters include the first passage speed and the speed at which the automatic guided vehicle moves in the navigable channel.

6. The production line modeling method based on digital twin according to claim 5, wherein the determining the speed of the automatic guided vehicle moving in the passable channel based on the distance between the automatic guided vehicle and the objects on the left and right sides and a fourth preset threshold value comprises: If the area of ​​the passable passage meets the first sub-preset threshold value, and the distance between the automatic guided vehicle and the objects on the left and right sides meets the second sub-preset threshold value, then determining that the speed of the automatic guided vehicle moving in the passable passage is the second pass speed; If the area of ​​the traversable passage does not satisfy the first sub-preset threshold, and / or the distance between the automatic guided vehicle and the objects on the left and right sides does not satisfy the second sub-preset threshold, the speed at which the automatic guided vehicle moves in the traversable passage is determined to be a third passing speed; and the second passing speed is higher than the third passing speed.

7. The production line modeling method based on digital twin according to any one of claims 1 to 3 or 6, further comprising: If the tilt angle of the automatic guided vehicle does not meet the angle threshold, the posture and speed of the automatic guided vehicle are adjusted based on the tilt angle; the environmental data includes the tilt angle, and the tilt angle includes the climbing angle of the automatic guided vehicle, the angle of deviation between the forward direction of the automatic guided vehicle and the center point of the navigable passage on the horizon; the traffic parameters include the posture and speed of the automatic guided vehicle; Based on the adjusted posture and speed of the automatic guided vehicle, the automatic guided vehicle is controlled to move to a position where the tilt angle meets the angle threshold.

8. According to any one of claims 1 to 3 or 6, the production line modeling method based on digital twins, in the process of controlling the automatic guided vehicle to move on the production line, collecting environmental data around the automatic guided vehicle, comprising: Controlling the automatic guided vehicle to obtain depth data of each object on the production line within the field of view of the depth camera through its own depth camera; Controlling the automatic guided vehicle to collect position data of each object on the production line through its own positioning device; Controlling the automatic guided vehicle to collect distance data of various objects on the production line through its own laser radar; The automatic guided vehicle is controlled to collect the tilt angle of the automatic guided vehicle through its own angle determination module.

9. According to any one of claims 1 to 3, or the method for modeling a production line based on digital twins according to claim 6, depth cameras are arranged around the automatic guided vehicle, and the depth cameras are used to collect depth data of each pixel point within the field of view; The determining the position of the object and the shape of the object based on the depth data and the position data comprises: Based on the position of the automatic guided vehicle, the depth data and position data of each pixel point around the automatic guided vehicle are integrated and processed to obtain the shape and position of objects around the automatic guided vehicle.

10. An electronic device, comprising: processor, memory, and communication bus; The communication bus is used to realize the communication connection between the processor and the memory; The processor is used to execute the production line modeling program based on digital twin in the memory to implement the following steps: In the process of controlling the automatic guided vehicle to move on the production line, environmental data around the automatic guided vehicle is collected; the environmental data at least includes: depth data and position data of each object on the production line; Determine a navigable passage, a navigable route and navigable parameters of the automated guided vehicle based on the depth data, the position data and preset conditions; Determining the position and shape of the object based on the depth data and the position data; The automatic guided vehicle is controlled to move according to the travel route and travel parameters, and a production line model of the production line is constructed based on the position and shape of the object.

Images