Frontal service platform intelligent manufacturing industry internet of things and control method

Abstract

The specification discloses a front formula service platform intelligent manufacturing industry internet of things, a control method and a medium, and relates to the technical field of intelligent manufacturing industry internet of things. The specification discloses the following: a obtaining module obtains an image captured by a first camera on a production line as first image data and an image captured by a second camera as second image data through a sensing network platform; the first camera and the second camera form a binocular camera; a three-dimensional module processes the image into three-dimensional image data; an identification module obtains a plurality of point positions of a distortion position as judgment point positions; a mapping module maps the judgment point positions into the second image data to form second calibration point positions; and a calibration module calibrates the first camera and the second camera according to the first calibration point positions and the second calibration point positions. The specification can realize timely correction of binocular camera calibration under the premise that the production line does not stop working, can reduce the influence of binocular camera calibration errors, can improve the accuracy of three-dimensional machine vision recognition, and can effectively improve production efficiency and equipment operation precision.

Description

Technical Field

[0001] This specification relates to the field of industrial Internet of Things (IoT) technology, and in particular to a front-end service platform for intelligent manufacturing of industrial IoT, control methods, and media. Background Technology

[0002] Machine vision is a rapidly developing branch of artificial intelligence. Simply put, machine vision uses machines to replace human eyes for measurement and judgment. A machine vision system uses machine vision products (i.e., image acquisition devices, which can be CMOS or CCD) to convert the captured target into image signals, which are then transmitted to a dedicated image processing system. This system obtains the target's shape information and, based on pixel distribution, brightness, color, and other information, converts it into digital signals. The image system performs various calculations on these signals to extract the target's features, and then controls the on-site equipment based on the judgment results.

[0003] The accuracy of machine vision is fundamental to equipment control in the Industrial Internet of Things (IIoT). To acquire visual images with depth, current technologies often use binocular cameras to obtain 3D images. Before use, binocular cameras require calibration. Calibration maps the coordinates of the images acquired by the two cameras in the binocular system, ensuring the target is imaged in the same coordinate system, thus obtaining depth information. However, in production lines, binocular cameras can introduce errors due to vibrations or other external factors, reducing 3D imaging accuracy and causing 3D image distortion. Summary of the Invention

[0004] In order to at least overcome the above-mentioned shortcomings in the prior art, the purpose of this specification is to provide a front-end service platform for intelligent manufacturing industrial IoT and control methods.

[0005] In a first aspect, embodiments of this specification provide a front-end distributed service platform for intelligent manufacturing industrial IoT, comprising a service platform, a management platform, and a sensor network platform that interact sequentially, wherein the management platform includes:

[0006] The acquisition module is configured to acquire images captured by a first camera on the production line as first image data and images captured by a second camera as second image data through the sensor network platform; the first camera and the second camera constitute a binocular camera.

[0007] A 3D module is configured to process the first image data and the second image data into 3D image data;

[0008] The recognition module is configured to acquire multiple points of the distorted part as judgment points when the distortion of the three-dimensional image data is detected;

[0009] The mapping module is configured to map the judgment point to the first image data to form a first calibration point, and to map the judgment point to the second image data to form a second calibration point;

[0010] The calibration module is configured to calibrate the first camera and the second camera based on the first calibration point and the second calibration point;

[0011] The service platform includes:

[0012] The service sub-platform is configured as multiple, and each receives calibration data from different binocular cameras;

[0013] The service platform is configured to aggregate all the aforementioned calibration data and display it through the user platform.

[0014] In existing technologies, the inventors discovered through scientific practice that the main method for calibrating binocular cameras is to set a checkerboard pattern in still images, specifically the Zhang Zhengyou calibration method. However, this method requires adding a calibration checkerboard pattern to the image, which means that the production line needs to be stopped for calibration, severely impacting production line efficiency.

[0015] In the implementation of the embodiments of this specification, the acquisition module can acquire the first image data and the second image data through traditional machine vision image acquisition methods, and generate the three-dimensional image data using existing techniques. However, the embodiments of this specification configure a recognition module to detect distortion in the three-dimensional image data. This detection process can be performed through image comparison or template comparison; this embodiment does not specify any limitations. When the three-dimensional image data is determined to be distorted, the recognition module can acquire the judgment points of the distorted areas. The number of judgment points can be determined according to the selected calibration algorithm. For example, for the Zhang Zhengyou calibration method, at least six sets of judgment points are required for calibration. The mapping module maps the corresponding judgment points to the first image data and the second image data respectively to obtain the first calibration point and the second calibration point for calibration. This allows for on-site binocular camera calibration via the calibration module, enabling calibration without stopping the production line. The embodiments in this specification, through the above-mentioned modules, can, on the one hand, realize timely correction of binocular camera calibration without stopping the production line, and on the other hand, reduce the impact of binocular camera calibration errors, improve the accuracy of 3D machine vision recognition, and effectively improve production efficiency and equipment operation precision.

[0016] Secondly, embodiments of this specification provide a front-end service platform intelligent manufacturing industrial Internet of Things control method, which is applied to a service platform, a management platform and a sensor network platform that interact sequentially. The service platform includes a main service platform and multiple sub-service platforms.

[0017] The control method executed by the management platform includes:

[0018] The sensor network platform acquires images captured by the first camera on the production line as first image data, and images captured by the second camera as second image data; the first camera and the second camera form a binocular camera.

[0019] The first image data and the second image data are processed into three-dimensional image data;

[0020] When distortion of the three-dimensional image data is detected, multiple points of the distorted area are obtained as judgment points;

[0021] The judgment points are mapped onto the first image data to form first calibration points, and the judgment points are mapped onto the second image data to form second calibration points;

[0022] The first camera and the second camera are calibrated based on the first calibration point and the second calibration point;

[0023] The service sub-platforms respectively receive calibration data from different binocular cameras;

[0024] The service platform aggregates all the calibration data and displays it through the user platform.

[0025] One embodiment of this specification provides a computer-readable storage medium that stores computer instructions. When a computer reads the computer instructions from the storage medium, the computer executes the intelligent manufacturing industrial Internet of Things control method of the front-end service platform.

[0026] Compared with existing technologies, this manual has the following advantages and beneficial effects: The intelligent manufacturing industrial IoT and control method of the front-end service platform in this manual can, on the one hand, realize timely correction of binocular camera calibration without stopping the production line, and on the other hand, reduce the impact of binocular camera calibration error, improve the accuracy of 3D machine vision recognition, and effectively improve production efficiency and equipment operation accuracy. Attached Figure Description

[0027] The accompanying drawings, which are included to provide a further understanding of the embodiments described herein and form part of this specification, do not constitute a limitation on the embodiments described herein. In the drawings;

[0028] Figure 1 This is a schematic diagram of the system architecture of an embodiment of this specification;

[0029] Figure 2 This is a schematic diagram of the method steps in the embodiments of this specification;

[0030] Figure 3 This is an exemplary flowchart illustrating the formation of a first calibration point and a second calibration point based on calibration values, according to some embodiments of this specification.

[0031] Figure 4a This is an exemplary schematic diagram illustrating the prediction of a first predicted value by a first prediction model according to some embodiments of this specification;

[0032] Figure 4b This is an exemplary schematic diagram illustrating the prediction of a second predicted value by a second prediction model according to some embodiments of this specification;

[0033] Figure 5 This is an exemplary schematic diagram illustrating the determination of a first calibration value and a second calibration value using a binocular calibration model, according to some embodiments of this specification. Detailed Implementation

[0034] To make the objectives, technical solutions, and advantages of the embodiments in this specification clearer, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the accompanying drawings in this specification are for illustrative and descriptive purposes only and are not intended to limit the scope of protection of this specification. Furthermore, it should be understood that the schematic drawings are not drawn to scale. The flowcharts used in this specification illustrate operations implemented according to some embodiments of this specification. It should be understood that the operations in the flowcharts may not be implemented in sequence, and steps without logical contextual relationships may be reversed or performed simultaneously. In addition, those skilled in the art, guided by the content of this specification, may add one or more other operations to the flowcharts, or remove one or more operations from the flowcharts.

[0035] Furthermore, the described embodiments are merely some, not all, of the embodiments described in this specification. The components of the embodiments described and illustrated in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this specification provided in the drawings is not intended to limit the scope of the claimed specification, but merely to illustrate selected embodiments of this specification. All other embodiments obtained by those skilled in the art based on the embodiments of this specification without inventive effort are within the scope of protection of this specification.

[0036] For a clearer explanation of the aforementioned front-end service platform for intelligent manufacturing and industrial IoT, please refer to the following references. Figure 1This specification provides a schematic diagram of the communication architecture of the front-end distributed service platform for intelligent manufacturing industrial IoT disclosed in the embodiments. The front-end distributed service platform for intelligent manufacturing industrial IoT may include a service platform, a management platform, and a sensor network platform that interact sequentially. The management platform includes:

[0037] The acquisition module is configured to acquire images captured by a first camera on the production line as first image data and images captured by a second camera as second image data through the sensor network platform; the first camera and the second camera constitute a binocular camera.

[0038] The first camera and the second camera refer to the video recording equipment used to capture images of the production line. The first camera and the second camera can include various types of video recording equipment, such as RGB cameras and infrared cameras. The first camera and the second camera can form a stereo camera, which comes in various types, such as stereo zoom cameras and stereo black light cameras.

[0039] The first image data refers to the data of the image captured by the first camera. The second image data refers to the data of the image captured by the second camera. The first and second image data may include data such as the size, color, and shape of the images.

[0040] A 3D module is configured to process the first image data and the second image data into 3D image data;

[0041] Three-dimensional image data refers to stereoscopic image data obtained by processing first and second image data. For example, three-dimensional image data may include data such as the volume, color, and shape of the image.

[0042] The recognition module is configured to acquire multiple points of the distorted part as judgment points when the distortion of the three-dimensional image data is detected;

[0043] In some embodiments, the judgment point can be represented by coordinates in the three-dimensional image data, for example, the coordinates of the judgment point are (7, 4, 0), etc. For a detailed explanation of distortion detection of three-dimensional image data and acquisition of judgment points of distorted parts, please refer to the relevant content on image comparison, template comparison and Zhang Zhengyou calibration method, etc.

[0044] The mapping module is configured to map the judgment point to the first image data to form a first calibration point, and to map the judgment point to the second image data to form a second calibration point;

[0045] The first calibration point refers to the corresponding point in the first image data, determined based on the calibration value of the mapping matrix between the three-dimensional image data and the first image data. In some embodiments, the first calibration point can be represented by coordinates within the first image data, such as coordinates (3, 2).

[0046] The second calibration point refers to the corresponding point in the second image data, determined based on the calibration value of the mapping matrix between the 3D image data and the second image data. In some embodiments, the second calibration point can be represented by coordinates within the second image data, such as coordinates (6, 4).

[0047] For definitions and explanations of the mapping matrix and calibration values, please refer to [link to documentation]. Figure 3 Related content.

[0048] In some embodiments, the mapping module is further configured to: obtain a mapping matrix, wherein the mapping matrix represents the mapping relationship between any two of the 3D image data, the first camera, and the second camera, and the mapping matrix includes a rotation matrix and a translation matrix; determine calibration values ​​based on the mapping matrix; and based on the calibration values, map the judgment points to the first image data to form first calibration points, and map the judgment points to the second image data to form second calibration points. For more details on the above, please refer to [link to relevant documentation]. Figure 3 Related descriptions.

[0049] The calibration module is configured to calibrate the first camera and the second camera based on the first calibration point and the second calibration point;

[0050] The service platform includes:

[0051] The service sub-platform is configured as multiple, and each receives calibration data from different binocular cameras;

[0052] The service platform is configured to aggregate all the aforementioned calibration data and display it through the user platform.

[0053] In existing technologies, the inventors discovered through scientific practice that the main method for calibrating binocular cameras is to set a checkerboard pattern in still images, specifically the Zhang Zhengyou calibration method. However, this method requires adding a calibration checkerboard pattern to the image, which means that the production line needs to be stopped for calibration, severely impacting production line efficiency.

[0054] In the implementation of the embodiments of this specification, the acquisition module can acquire the first image data and the second image data through traditional machine vision image acquisition methods, and generate the three-dimensional image data using existing techniques. However, the embodiments of this specification configure a recognition module to detect distortion in the three-dimensional image data. This detection process can be performed through image comparison or template comparison; this embodiment does not specify any limitations. When the three-dimensional image data is determined to be distorted, the recognition module can acquire the judgment points of the distorted areas. The number of judgment points can be determined according to the selected calibration algorithm. For example, for the Zhang Zhengyou calibration method, at least six sets of judgment points are required for calibration. The mapping module maps the corresponding judgment points to the first image data and the second image data respectively to obtain the first calibration point and the second calibration point for calibration. This allows for on-site binocular camera calibration via the calibration module, enabling calibration without stopping the production line. The embodiments in this specification, through the above-mentioned modules, can, on the one hand, realize timely correction of binocular camera calibration without stopping the production line, and on the other hand, reduce the impact of binocular camera calibration errors, improve the accuracy of 3D machine vision recognition, and effectively improve production efficiency and equipment operation precision.

[0055] In one possible implementation, the identification module is further configured as follows:

[0056] The system detects whether the three-dimensional image data is distorted based on a preset template.

[0057] When distortion of the three-dimensional image data is detected, a region of interest is established centered on the point of maximum distortion.

[0058] Based on the preset template, at least two consecutive distorted edges are identified from the region of interest. If no at least two consecutive distorted edges are identified, the region of interest is redefined and identified again.

[0059] According to a preset template, separate edges and shared edges are identified from the distorted edges, and multiple points are extracted from the separate edges as first points and multiple points are extracted from the shared edges as second points; the separate edges are the edges where the same points in the first image data and the second image data do not overlap in the three-dimensional image data; the shared edges are the edges where the same points in the first image data and the second image data overlap in the three-dimensional image data.

[0060] The first point and the second point are used as the judgment points.

[0061] In implementing the embodiments of this specification, the inventors discovered that when distortion occurs, the distorted area is mainly manifested as two non-overlapping lines appearing where the area should be a single line in the 3D image data. Therefore, the inventors adopted a preset template method for distortion judgment and identification of distorted areas. The preset template generally uses a type of 3D image data, and the preset template can establish edges for identification based on fixed edges in the 3D image data. These edges can be used to identify distorted edges. To effectively improve the accuracy of calibration, the embodiments of this specification divide the identified distorted edges into separated edges and shared edges. Separated edges are those edges that have been distorted, while shared edges are those edges within a predetermined range of the separated edges that have not been distorted. Because separated edges and shared edges are set separately, in the subsequent calibration process, closed calibration can be performed based on the characteristics of separated edges and shared edges, effectively improving the accuracy of the calibration.

[0062] In one possible implementation, the calibration module is further configured as follows:

[0063] When calibrating the first camera and the second camera based on the first calibration point and the second calibration point, the calibration matrix for generating the 3D image data is inverted based on the first verification point, and the inversion result is verified based on the second verification point to form the adjustment parameters for the first camera and / or the second camera; the first verification point is the first calibration point and the second calibration point generated by mapping the first point; the second verification point is the first calibration point and the second calibration point generated by mapping the second point;

[0064] The first camera and / or the second camera are calibrated according to the adjustment parameters.

[0065] In the implementation of the embodiments of this specification, data inversion is performed through at least the first verification point, that is, the coordinates are inverted through the calibration matrix, and the adjustment relationship that the first camera and the second camera should complete can be calculated. The adjustment relationship can be verified through the second verification point, which has no distortion, so that the inversion at the first verification point will not overfit, effectively improving the fitting accuracy.

[0066] It should be understood that the calibration matrix is ​​generated during the initial calibration of the stereo camera. It is used to characterize the mapping relationship between the image coordinates captured by the stereo camera and the coordinates in the 3D image data. For different initial calibration methods, the corresponding calibration matrix is ​​different, and those skilled in the art can choose the calibration matrix as needed. For example, for the Zhang Zhengyou calibration method, the calibration matrix can be an intrinsic parameter matrix and an extrinsic parameter matrix.

[0067] In one possible implementation, the preset template is provided with multiple recognition edges;

[0068] The recognition module is also configured as follows:

[0069] The preset template is periodically compared with the three-dimensional image data, and the three-dimensional image data is judged to be distorted when the difference between any recognized edge and the corresponding edge data in the three-dimensional image data is greater than a preset value.

[0070] In the implementation of the embodiments of this specification, the recognition edges set in the preset template are used as the basis for distortion judgment in the embodiments of this specification. Through periodic recognition, it is possible to determine whether the three-dimensional image data is distorted. Where the difference between the recognition edge and the edge data is greater than a preset value, existing line comparison mechanisms, such as fitting, can be used to achieve this, which will not be repeated here.

[0071] In one possible implementation, the identification module is further configured as follows:

[0072] Based on the recognition edge in the preset template, the distorted edge is identified from the region of interest as a separating edge, and the separating edge is extended along the recognition edge to form a shared edge.

[0073] In the implementation of the embodiments of this specification, the separation edge is identified by recognizing the edge, and the endpoint of the separation edge is extended along the recognized edge to form a common edge, thereby obtaining the first point and the second point, thereby improving the accuracy of detection.

[0074] In some embodiments, the mapping module can also be used to obtain a mapping matrix, wherein the mapping matrix represents the mapping relationship between any two of the three-dimensional image data, the first camera, and the second camera, and the mapping matrix includes a rotation matrix and a translation matrix; based on the mapping matrix, a calibration value is determined; based on the calibration value, the judgment point is mapped to the first image data to form a first calibration point, and the judgment point is mapped to the second image data to form a second calibration point.

[0075] In some embodiments, the mapping module can also be used to determine the predicted value of the mapping matrix based on the mapping matrix, and to determine the calibration value of the mapping matrix based on the predicted value.

[0076] In some embodiments, the mapping module can also be used to predict a first predicted value of the first mapping matrix based on the first mapping matrix using a first prediction model, wherein the first prediction model is a machine learning model and the input of the first prediction model includes a first theoretical value of the first mapping matrix, the conveyor belt speed, and motion sensor data; and to predict a second predicted value of the second mapping matrix based on the second mapping matrix using a second prediction model, wherein the second prediction model is a machine learning model and the input of the second prediction model includes a second theoretical value of the second mapping matrix, the conveyor belt speed, and motion sensor data.

[0077] In some embodiments, the mapping module can also be used to determine a first calibration value of the first mapping matrix and a second calibration value of the second mapping matrix based on the first predicted value and the second predicted value through a binocular calibration model. The binocular calibration model is a machine learning model. The first calibration value includes a first rotation matrix calibration value and a first translation matrix calibration value. The second calibration value includes a second rotation matrix calibration value and a second translation matrix calibration value. The first calibration value is used to map the judgment point to the first image data to form a first calibration point. The second calibration value is used to map the judgment point to the second image data to form a second calibration point.

[0078] In some embodiments, the training of the first prediction model, the second prediction model, and the binocular calibration model includes joint training.

[0079] In some embodiments, the input of the binocular calibration model includes reference point information, wherein the reference point information includes at least one of a first position and a second position of the reference point, wherein the reference point is a preset point in the three-dimensional image data, the first position is the position of the reference point on the first image data, and the second position is the position of the reference point on the second image data.

[0080] In some embodiments, the reference point information further includes an offset, wherein the offset is the vector difference between the first position vector and the second position vector of the second position, wherein the first position vector is the position vector of the first position mapped onto the second image data based on the stereo mapping matrix of the stereo prediction value of the first position, and the stereo mapping matrix is ​​the mapping between the first camera and the second camera.

[0081] In some embodiments, the mapping module can also be used to determine the stereo prediction value based on the first prediction value and the second prediction value.

[0082] Based on the above, please refer to the following: Figure 2 This is a flowchart illustrating the intelligent manufacturing industrial IoT control method for a front-end service platform provided in the embodiments of this specification. The intelligent manufacturing industrial IoT control method for a front-end service platform can be applied to… Figure 1 The front-end service platform for intelligent manufacturing industrial IoT further includes, in detail, the control method for the front-end service platform for intelligent manufacturing industrial IoT, which may include the content described in steps S1-S5.

[0083] S1: The image captured by the first camera on the production line is used as the first image data, and the image captured by the second camera is used as the second image data through the sensor network platform; the first camera and the second camera form a binocular camera.

[0084] S2: Process the first image data and the second image data into three-dimensional image data;

[0085] S3: When distortion of the three-dimensional image data is detected, multiple points of the distorted part are obtained as judgment points;

[0086] S4: Map the judgment point to the first image data to form a first calibration point, and map the judgment point to the second image data to form a second calibration point;

[0087] S5: Calibrate the first camera and the second camera according to the first calibration point and the second calibration point.

[0088] In one possible implementation, when distortion of the three-dimensional image data is detected, acquiring multiple points of the distorted area as judgment points includes:

[0089] The system detects whether the three-dimensional image data is distorted based on a preset template.

[0090] When distortion of the three-dimensional image data is detected, a region of interest is established centered on the point of maximum distortion.

[0091] Based on the preset template, at least two consecutive distorted edges are identified from the region of interest. If no at least two consecutive distorted edges are identified, the region of interest is redefined and identified again.

[0092] According to a preset template, separate edges and shared edges are identified from the distorted edges, and multiple points are extracted from the separate edges as first points and multiple points are extracted from the shared edges as second points; the separate edges are the edges where the same points in the first image data and the second image data do not overlap in the three-dimensional image data; the shared edges are the edges where the same points in the first image data and the second image data overlap in the three-dimensional image data.

[0093] The first point and the second point are used as the judgment points.

[0094] In one possible implementation, calibrating the first camera and the second camera based on the first calibration point and the second calibration point includes:

[0095] When calibrating the first camera and the second camera based on the first calibration point and the second calibration point, the calibration matrix for generating the 3D image data is inverted based on the first verification point, and the inversion result is verified based on the second verification point to form the adjustment parameters for the first camera and / or the second camera; the first verification point is the first calibration point and the second calibration point generated by mapping the first point; the second verification point is the first calibration point and the second calibration point generated by mapping the second point;

[0096] The first camera and / or the second camera are calibrated according to the adjustment parameters.

[0097] In one possible implementation, the preset template is provided with multiple recognition edges;

[0098] Detecting whether the three-dimensional image data is distorted according to a preset template includes:

[0099] The preset template is periodically compared with the three-dimensional image data, and the three-dimensional image data is judged to be distorted when the difference between any recognized edge and the corresponding edge data in the three-dimensional image data is greater than a preset value.

[0100] In one possible implementation, identifying separating edges and shared edges from the distorted edges based on a preset template includes:

[0101] Based on the recognition edge in the preset template, the distorted edge is identified from the region of interest as a separating edge, and the separating edge is extended along the recognition edge to form a shared edge.

[0102] Figure 3 This is an exemplary flowchart illustrating the formation of a first calibration point and a second calibration point based on calibration values, according to some embodiments of this specification. Figure 3 As shown, process 300 includes the following steps. In some embodiments, process 300 may be executed by a mapping module.

[0103] Step 310: Obtain the mapping matrix, which represents the mapping relationship between any two of the 3D image data, the first camera, and the second camera. The mapping matrix includes a rotation matrix and a translation matrix.

[0104] For definitions and explanations of 3D image data, the first camera, and the second camera, please refer to [link to relevant documentation]. Figure 1 Related content.

[0105] A mapping matrix can represent the mapping relationship between any two of the following: 3D image data, a first camera, and a second camera. A mapping matrix can include a mapping matrix between 3D image data and the first camera, a mapping matrix between 3D image data and the second camera, and a mapping matrix between the first camera and the second camera. Each element in the mapping matrix can reflect the coordinate relationship at the same location between 3D image data and the first image data, or between 3D image data and the second image data, or between the first image data and the second image data.

[0106] In some embodiments, the mapping matrix may include rotation and translation matrices. For example, the mapping matrix between 3D image data and a first camera (e.g., a first mapping matrix) may include a first rotation matrix R1 mapping the 3D image data to the first image data and a first translation matrix T1 mapping the 3D image data to the first image data. The first rotation matrix R1 describes the orientation of the coordinate axes of the world coordinate system relative to the coordinate axes of the first camera. The first translation matrix T1 describes the position of the origin of space in the first camera coordinate system. As another example, the mapping matrix between 3D image data and a second camera (e.g., a second mapping matrix) may include a second rotation matrix R2 mapping the 3D image data to the second image data and a second translation matrix T2 mapping the 3D image data to the second image data. The second rotation matrix R2 describes the orientation of the coordinate axes of the world coordinate system relative to the coordinate axes of the second camera. The second translation matrix T2 describes the position of the origin of space in the second camera coordinate system. As yet another example, the mapping matrix between a first camera and a second camera (e.g., a stereo mapping matrix) may include a stereo rotation matrix R mapping the first image data to the second image data and a stereo translation matrix T mapping the first image data to the second image data. The stereo rotation matrix R describes the orientation of the first camera's coordinate axes relative to the second camera's coordinate axes. The stereo translation matrix T describes the position of the origin of the first camera's coordinate system in the second camera's coordinate system.

[0107] In some embodiments, the mapping module can obtain the mapping matrix in various ways. For example, the mapping module can calculate the mapping matrix using an algorithm. Exemplarily, the mapping module can calculate the mapping matrix between the 3D image data and the first camera, and the mapping matrix between the 3D image data and the second camera using a mapping algorithm. The mapping module can calculate the mapping matrix between the first camera and the second camera according to formulas. The formula for calculating the binocular rotation matrix R can be expressed as R = R² * R₁^T. The formula for calculating the binocular translation matrix T can be expressed as T = T² * T₁^T, where ^T represents the transpose of the matrix.

[0108] Step 320: Determine the calibration value based on the mapping matrix.

[0109] The calibration value can refer to the calibrated mapping matrix. The mapping module can adjust the mapping matrix based on calibration parameters to determine the calibration value. For example, after adjusting the mapping matrix A between the 3D image data and the first camera, matrix (wA) is obtained. Here, w is the calibration parameter of the mapping matrix A, and the calibration value of the mapping matrix between the 3D image data and the first camera is matrix (wA).

[0110] The mapping module can determine the calibration value of the mapping matrix in various ways. In some embodiments, the mapping module can determine the calibration value of the mapping matrix based on historical data. For example, the mapping module can obtain the calibration value of the historical time point closest to the current time point in the historical mapping matrix as the calibration value of the current mapping matrix.

[0111] In some embodiments, the mapping module may determine the predicted value of the mapping matrix based on the mapping matrix. Based on the predicted value, the calibration value of the mapping matrix may be determined.

[0112] The predicted value of the mapping matrix refers to the mapping matrix obtained by considering factors such as production line vibration. For example, after predicting the mapping matrix B between 3D image data and the second camera, matrix (kB) is obtained. Here, k is the prediction parameter of mapping matrix B, and the predicted value of the mapping matrix between 3D image data and the second camera is matrix (kB).

[0113] The mapping module can determine the predicted value of the mapping matrix in various ways. In some embodiments, the mapping module can analyze and process the relationship between factors such as production line vibration and the mapping matrix to determine the predicted value of the mapping matrix. For example, the change function of the mapping matrix affected by production line vibration can be determined based on historical data, and the predicted value of the current mapping matrix can be determined by function calculation based on the current production line vibration situation and the current mapping matrix.

[0114] In some embodiments, the mapping module can determine different predicted values ​​for different mapping matrices using either a first prediction model or a second prediction model. For example, the mapping module can predict a first predicted value for the first mapping matrix using the first prediction model. As another example, the mapping module can predict a second predicted value for the second mapping matrix using the second prediction model. For a detailed explanation of determining different predicted values ​​for different mapping matrices using either the first or second prediction model, see [link to documentation]. Figure 4a or Figure 4b And related content.

[0115] In some embodiments, the mapping module can determine the calibration value of the mapping matrix based on the predicted values ​​in various ways. For example, the mapping module can analyze and process historical predicted values ​​and corresponding historical calibration values ​​to establish a relationship function between the predicted values ​​and corresponding calibration values, thereby determining the calibration value of the current mapping matrix. As another example, the mapping module can also determine the first calibration value of the first mapping matrix and the second calibration value of the second mapping matrix using a binocular calibration model. For a detailed explanation of determining the first calibration value of the first mapping matrix and the second calibration value of the second mapping matrix using a binocular calibration model, please refer to [link to relevant documentation]. Figure 5 And related content.

[0116] Determining the calibration value of the mapping matrix based on the predicted value allows for a comprehensive consideration of the impact of factors such as production line vibration on the mapping matrix, thus enabling a more accurate determination of the calibration value.

[0117] Step 330: Based on the calibration value, the judgment point is mapped to the first image data to form the first calibration point, and the judgment point is mapped to the second image data to form the second calibration point.

[0118] Judgment points refer to multiple points obtained from distorted areas of 3D image data. For detailed explanations of judgment points, first calibration points, and second calibration points, please refer to [link to relevant documentation]. Figure 1 Related content.

[0119] Mapping the judgment point to the first image data and the second image data means determining the corresponding point in the first image data and the second image data based on the calibration value of the mapping matrix between the three-dimensional image data and the first image data and the second image data.

[0120] In some embodiments, the mapping module can calculate the first and second calibration points corresponding to the judgment point using a formula based on the calibration value. For example, the first and second calibration points can be determined based on the coordinate transformation formula X*H = Y. Here, X represents the coordinates of the judgment point in the 3D image data. H represents the calibration value of the mapping matrix between the 3D image data and the first image data, or the calibration value of the mapping matrix between the 3D image data and the second image data. When H represents the calibration value of the mapping matrix between the 3D image data and the first image data, Y represents the coordinates of the first calibration point mapped from the judgment point to the first calibration point in the first image data; when H represents the calibration value of the mapping matrix between the 3D image data and the second image data, Y represents the coordinates of the second calibration point mapped from the judgment point to the second calibration point in the second image data.

[0121] By determining the calibration points in the first and second image data based on the calibration values ​​of the mapping matrix, the adjustment parameters of the binocular camera can be accurately determined, thereby improving the accuracy of visual recognition of the binocular camera and effectively improving production efficiency and equipment operation precision.

[0122] In some embodiments, the mapping matrix may include a first mapping matrix and a second mapping matrix. The first mapping matrix may be a mapping between 3D image data and a first camera. The second mapping matrix may be a mapping between 3D image data and a second camera. More information on the mapping between 3D image data and the first or second camera can be found in [link to relevant documentation]. Figure 3 And its related descriptions.

[0123] Figure 4a This is an exemplary schematic diagram illustrating the prediction of a first predicted value by a first prediction model according to some embodiments of this specification.

[0124] In some embodiments, based on the first mapping matrix, the mapping module can predict the first predicted value of the first mapping matrix using a first prediction model.

[0125] The first prediction model 421 can be a machine learning model. In some embodiments, the type of the first prediction model may include a neural network model, etc., and the choice of model type may depend on the specific circumstances.

[0126] In some embodiments, the inputs to the first prediction model may include a first theoretical value 411 of the first mapping matrix, a conveyor belt speed 412, motion sensor data 413, etc. The output of the first prediction model may include a first predicted value 431 of the first mapping matrix.

[0127] The first theoretical value can refer to the first mapping matrix that maps 3D image data to the first image data without the influence of other factors. Other factors may include production line vibration, other external factors, etc. The first mapping matrix may include a first rotation matrix and a first translation matrix. More information about the first rotation matrix and the first translation matrix can be found in [link to relevant documentation]. Figure 3 Related descriptions.

[0128] In some embodiments, the mapping module can determine the first rotation matrix and the first translation matrix corresponding to the previous (e.g., the (n-1)th) calibration of the first camera as the first theoretical value of the first mapping matrix corresponding to the current (e.g., the nth) calibration. When the mapping module performs the next (e.g., the (n+1)th) calibration, the mapping module can determine the first rotation matrix and the first translation matrix corresponding to the current (e.g., the nth) calibration as the first theoretical value of the first mapping matrix corresponding to the next (e.g., the (n+1)th) calibration.

[0129] The conveyor belt speed can be determined by the operating parameters set for the conveyor belt. For example, the conveyor belt speed can be set to 2.5 m / s.

[0130] Motion sensor data refers to data acquired through motion sensors. This data can include the motion posture, acceleration, and vibration intensity of the object being detected. Examples include acceleration in the x, y, and z directions, attitude angles in the Rx, Ry, and Rz directions, amplitude, and vibration frequency. The object being detected can refer to products manufactured on a production line.

[0131] The first predicted value can be the predicted value of the first mapping matrix. The first predicted value can include the predicted value of the first rotation matrix and the predicted value of the first translation matrix. The predicted value of the first rotation matrix can reflect the rotation matrix that maps the 3D image data to the first image data after considering factors such as production line vibration, and the predicted value of the first translation matrix can reflect the translation matrix that maps the 3D image data to the first image data after considering factors such as production line vibration.

[0132] In some embodiments, the mapping module can obtain a first prediction model, a second prediction model, and a stereo calibration model through joint training. More information on joint training can be found in [link to relevant documentation]. Figure 5 Related descriptions.

[0133] Figure 4b This is an exemplary schematic diagram illustrating the prediction of a second predicted value using a second prediction model, according to some embodiments of this specification.

[0134] In some embodiments, based on the second mapping matrix, the mapping module can predict the second predicted value of the second mapping matrix using the second prediction model.

[0135] The second prediction model 422 can be a machine learning model. In some embodiments, the type of the second prediction model may include a neural network model, etc., and the choice of model type may depend on the specific circumstances.

[0136] In some embodiments, the inputs to the second prediction model may include a second theoretical value 414 of the second mapping matrix, a conveyor belt speed 412, motion sensor data 413, etc. The output of the second prediction model may include a second predicted value 432 of the second mapping matrix.

[0137] The second theoretical value can refer to the second mapping matrix that maps 3D image data to second image data without the influence of other factors. Other factors may include production line vibration, other external factors, etc. The second mapping matrix may include a second rotation matrix and a second translation matrix. For more information on the second rotation matrix and the second translation matrix, please refer to... Figure 3 Related descriptions.

[0138] In some embodiments, the mapping module can determine the second rotation matrix and the second translation matrix corresponding to the previous (e.g., the (n-1)th) calibration of the second camera as the second theoretical value of the second mapping matrix corresponding to the current (e.g., the nth) calibration. When the mapping module performs the next (e.g., the (n+1)th) calibration, the mapping module can determine the second rotation matrix and the second translation matrix corresponding to the current (e.g., the nth) calibration as the second theoretical value of the second mapping matrix corresponding to the next (e.g., the (n+1)th) calibration.

[0139] The second predicted value can be the predicted value of the second mapping matrix. The second predicted value can include the predicted value of the second rotation matrix and the predicted value of the second translation matrix. The predicted value of the second rotation matrix can reflect the rotation matrix mapping the 3D image data to the second image data, taking into account factors such as production line vibration. The predicted value of the second translation matrix can reflect the translation matrix mapping the 3D image data to the second image data, taking into account factors such as production line vibration.

[0140] In some embodiments, the mapping module can obtain a first prediction model, a second prediction model, and a stereo calibration model through joint training. More information on joint training can be found in [link to relevant documentation]. Figure 5 Related descriptions.

[0141] By predicting the first and second predicted values ​​using the first and second prediction models respectively, factors such as production line vibration can be comprehensively considered. This allows for a more accurate determination of the first and second predicted values ​​of the first and second mapping matrices, which is more conducive to improving the accuracy of 3D machine vision recognition and effectively improving production efficiency and equipment operation precision.

[0142] Figure 5 This is an exemplary schematic diagram illustrating the determination of a first calibration value and a second calibration value using a binocular calibration model, according to some embodiments of this specification. Figure 5 As shown, determining the first calibration value and the second calibration value using a binocular calibration model includes the following. In some embodiments, Figure 5 The content shown can be completed by the mapping module.

[0143] In some embodiments, the calibration values ​​include a first calibration value and a second calibration value. The mapping module can determine the first calibration value of the first mapping matrix and the second calibration value of the second mapping matrix based on the first and second predicted values ​​using a binocular calibration model. The binocular calibration model is a machine learning model. The first calibration value includes a first rotation matrix calibration value and a first translation matrix calibration value. The second calibration value includes a second rotation matrix calibration value and a second translation matrix calibration value. The first calibration value is used to map the judgment point to the first image data to form a first calibration point, and the second calibration value is used to map the judgment point to the second image data to form a second calibration point.

[0144] The first calibration value refers to the calibration value corresponding to the first mapping matrix. The first calibration value may include the first rotation matrix calibration value and the first translation matrix calibration value. The first rotation matrix calibration value refers to the calibration value of the rotation matrix mapping the 3D image data to the first image data. The first translation matrix calibration value refers to the calibration value of the translation matrix mapping the 3D image data to the first image data. The first calibration value is used to map the judgment point to the first image data to form the first calibration point. For more information on the first calibration value, please refer to [link to documentation]. Figure 3 Related content.

[0145] The second calibration value refers to the calibration value corresponding to the second mapping matrix. The second calibration value can include the second rotation matrix calibration value and the second translation matrix calibration value. The second rotation matrix calibration value refers to the calibration value of the rotation matrix mapping the 3D image data to the second image data. The second translation matrix calibration value refers to the calibration value of the translation matrix mapping the 3D image data to the second image data. The second calibration value is used to map the judgment point to the second image data to form the second calibration point. For more information on the second calibration value, please refer to [link to documentation]. Figure 3 Related content.

[0146] The binocular calibration model 520 is used to calibrate the first mapping matrix and the second mapping matrix by combining reference point information. In some embodiments, the binocular calibration model can be a trained machine learning model. For example, the binocular calibration model can be a DNN (Deep Neural Networks) model, etc.

[0147] The input 510 to the binocular calibration model may include a first predicted value 431 of the first mapping matrix, a second predicted value 432 of the second mapping matrix, the conveyor belt speed, and motion sensor data, etc. Regarding the conveyor belt speed and motion sensor data... Figure 5 Not shown in the diagram. The output 530 of the binocular calibration model can be the first calibration value 531 of the first mapping matrix and the second calibration value 532 of the second mapping matrix. For definitions and explanations of the first predicted value 431 of the first mapping matrix, the second predicted value 432 of the second mapping matrix, the conveyor belt speed, and the motion sensor data, please refer to [link to documentation]. Figure 4a or Figure 4b Related content.

[0148] In some embodiments, the input 510 of the binocular calibration model may further include reference point information 511. The reference point information includes at least one of a first position and a second position of the reference point, wherein the reference point is a preset point in the three-dimensional image data, the first position is the position of the reference point on the first image data, and the second position is the position of the reference point on the second image data.

[0149] Reference points are preset points in the 3D image data. Preset points refer to easily measurable landmark points in the 3D image data. For example, preset points include the center point, highest point, and lowest point of the 3D image data.

[0150] In some embodiments, the reference point information may include at least one of a first position and a second position of the reference point. For example, for multiple reference points a, b, and c, the reference point information may include the first position of reference point a, the second position of reference point b, and the first and second positions of reference point c, etc.

[0151] The first position is the location of the reference point on the first image data. For example, the first position is the actual location of the reference point on the first image data. In some embodiments, the first position can be obtained by manual measurement.

[0152] The second position is the location of the reference point on the second image data. For example, the second position is the actual location of the reference point on the second image data. In some embodiments, the second position can be obtained by manual measurement.

[0153] Inputting reference point information into the binocular calibration model allows the reference point information to be used to assist in determining calibration values, making the determination of calibration values ​​more accurate.

[0154] In some embodiments, the reference point information 511 further includes an offset. The offset is the vector difference between the first position vector and the second position vector of the second position, wherein the first position vector is the position vector mapped from the binocular prediction value of the first position based on the binocular mapping matrix onto the second image data, and the binocular mapping matrix is ​​the mapping between the first camera and the second camera.

[0155] The offset is the vector difference between the first position vector and the second position vector.

[0156] The first position vector is the position vector on the second image data mapped from the stereo prediction value based on the stereo mapping matrix. The second position vector is the vector representing the position of the reference point on the second image data.

[0157] For example, if the reference point is located at a first position P (3, 2) in the first image data, and the first position P' is mapped onto the second image data based on the stereo prediction value of the stereo mapping matrix, and the origin of the coordinate system of the second image data is O, then a vector can be constructed. As the first position vector. If the reference point is located at the second position P” in the second image data and is (2, 4), then a vector can be constructed. As the second position vector, the vector difference between the first and second position vectors can be expressed as:

[0158] The stereo mapping matrix is ​​the mapping matrix between the first camera and the second camera. For more information on the mapping matrix between the first and second cameras, please refer to [link to documentation / reference]. Figure 3 Related content.

[0159] The stereo prediction value refers to the predicted value of the stereo mapping matrix. For example, after predicting the stereo mapping matrix C, we obtain matrix (sC). Here, s is the prediction parameter of the stereo mapping matrix C, and the predicted value of the stereo prediction matrix is ​​matrix (sC).

[0160] In some embodiments, the mapping module can obtain stereo prediction values ​​based on historical data. For example, the mapping module can determine the stereo prediction value with the closest historical time point to the current time point from the historical stereo prediction values ​​as the current stereo prediction value.

[0161] In some embodiments, the mapping module can also determine the stereo prediction value based on the first prediction value and the second prediction value. For example, the stereo prediction value can be determined based on the calculation formula between the first prediction value of the first mapping matrix, the second prediction value of the second mapping matrix, and the stereo prediction value of the stereo mapping matrix (such as the calculation formula for the stereo rotation matrix R and the stereo translation matrix T). For specific calculation methods, please refer to [link to relevant documentation]. Figure 3 Related descriptions.

[0162] Based on the first and second predicted values, the binocular predicted values ​​can be determined through quantitative calculations, making the obtained binocular predicted values ​​more accurate. This allows for a more accurate acquisition of the first position vector, thereby improving the accuracy of the calibration values.

[0163] By inputting the offset as reference point information into the binocular calibration model, deviations caused by factors other than production line vibration can be fully considered, resulting in more accurate calibration values.

[0164] In some embodiments, the training of the first prediction model, the second prediction model, and the binocular calibration model includes joint training.

[0165] In some embodiments, the mapping module can train a first prediction model, a second prediction model, and a stereo calibration model based on a large number of labeled training samples. Each set of training samples may include a first theoretical value, a second theoretical value, a sample conveyor belt speed, sample motion sensor data, and sample reference point information, etc. The label for each set of training samples is the first calibration value of the first sample mapping matrix and the second calibration value of the second sample mapping matrix corresponding to that set of training samples. In some embodiments, the multiple sets of training samples and labels may be derived from historical data. For example, the sample conveyor belt speed and sample motion sensor data may be derived from historical production line data. The sample reference point information may be derived from historical reference point information. In some embodiments, the label for each set of training samples may be derived from historical data, such as historical calibration records corresponding to each set of training samples.

[0166] During joint training, the mapping module inputs the first theoretical value of the sample, the conveyor belt speed, and the motion sensor data from each group of labeled training samples into the initial first prediction model to obtain the first predicted value of the first mapping matrix output by the initial first prediction model. The mapping module then inputs the second theoretical value of the sample, the conveyor belt speed, and the motion sensor data from the same group of training samples into the initial second prediction model to obtain the second predicted value of the second mapping matrix output by the initial second prediction model. Finally, the mapping module inputs the first predicted value of the first mapping matrix output by the initial first prediction model, the second predicted value of the second mapping matrix output by the initial second prediction model, the conveyor belt speed, motion sensor data, and reference point information from the same group of training samples into the initial stereo calibration model to obtain the first calibration value of the first mapping matrix and the second calibration value of the second mapping matrix. A loss function is constructed based on the output and labels of the stereo calibration model. The parameters of the first prediction model, the second prediction model, and the stereo calibration model are updated based on the loss function to obtain the trained first prediction model, the second prediction model, and the stereo calibration model.

[0167] In some embodiments of this specification, joint training of the first prediction model, the second prediction model, and the stereo calibration model helps to address the difficulty in obtaining labels when training these models individually. This reduces the required number of samples and also improves training efficiency. In some embodiments, joint training can lead to higher accuracy in the outputs of the trained first prediction model, second prediction model, and stereo calibration model, thereby obtaining more precise first and second calibration values.

[0168] In some embodiments of this specification, the first calibration value and the second calibration value are determined by combining the reference point, the first predicted value, and the second predicted value. This not only fully considers the influence of production line vibration factors but also reduces deviations caused by factors other than production line vibration factors. This results in a more accurate calibrated mapping matrix, which is more conducive to improving the accuracy of 3D machine vision recognition and effectively improving production efficiency and equipment operating precision.

[0169] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this specification.

[0170] In the several embodiments provided in this specification, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the couplings or direct couplings or communication connections shown or discussed may be indirect couplings or communication connections through some interfaces, devices, or units, or they may be electrical, mechanical, or other forms of connection.

[0171] The units described as separate components may or may not be physically separate. As will be apparent to those skilled in the art, the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of each example have been generally described in terms of function in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this specification.

[0172] Furthermore, the functional units in the various embodiments of this specification can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0173] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this specification, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or grid device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this specification. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0174] The specific embodiments described above further illustrate the purpose, technical solutions, and beneficial effects of this specification. It should be understood that the above descriptions are merely specific embodiments of this specification and are not intended to limit the scope of protection of this specification. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this specification should be included within the scope of protection of this specification.

Claims

1. A front-end distributed service platform for intelligent manufacturing industrial IoT, comprising a service platform, a management platform, and a sensor network platform that interact sequentially, characterized in that: The management platform includes: an acquisition module configured to acquire images captured by a first camera on the production line as first image data and images captured by a second camera as second image data through the sensor network platform; the first camera and the second camera constitute a binocular camera. The three-dimensional module is configured to process the first image data and the second image data into three-dimensional image data; The recognition module is configured to acquire multiple points of the distorted part as judgment points when distortion of the three-dimensional image data is detected; The mapping module is configured to map the judgment point to the first image data to form a first calibration point, and to map the judgment point to the second image data to form a second calibration point; The calibration module is configured to calibrate the first camera and the second camera based on the first calibration point and the second calibration point; The service platform includes: The service sub-platform is configured as multiple, and each receives calibration data from different binocular cameras; The service platform is configured to aggregate all the calibration data and display it through the user platform; The recognition module is also configured to: detect whether the three-dimensional image data is distorted according to a preset template; When distortion of the three-dimensional image data is detected, a region of interest is established centered on the point of maximum distortion. Based on the preset template, at least two consecutive distorted edges are identified from the region of interest. If no at least two consecutive distorted edges are identified, the region of interest is redefined and identified again. According to a preset template, separate edges and shared edges are identified from the distorted edges, and multiple points are extracted from the separate edges as first points and multiple points are extracted from the shared edges as second points; the separate edges are the edges where the same points in the first image data and the second image data do not overlap in the three-dimensional image data; the shared edges are the edges where the same points in the first image data and the second image data overlap in the three-dimensional image data. The first point and the second point are used as the judgment points.

2. The intelligent manufacturing industrial IoT platform for front-end distributed services according to claim 1, characterized in that, The calibration module is further configured to: when calibrating the first camera and the second camera according to the first calibration point and the second calibration point, invert the calibration matrix that generates the three-dimensional image data according to the first verification point, and verify the inversion result according to the second verification point to form the adjustment parameters of the first camera and / or the second camera; The first verification point is the first calibration point and the second calibration point generated by mapping the first point; The second verification point is the first calibration point and the second calibration point generated by mapping the second point. The first camera and / or the second camera are calibrated according to the adjustment parameters. The preset template is configured with multiple recognition edges; The recognition module is also configured as follows: The preset template is periodically compared with the three-dimensional image data, and the three-dimensional image data is judged to be distorted when the difference between any recognized edge and the corresponding edge data in the three-dimensional image data is greater than a preset value. The identification module is also configured to: Based on the recognition edge in the preset template, the distorted edge is identified from the region of interest as a separating edge, and the separating edge is extended along the recognition edge to form a shared edge.

3. The intelligent manufacturing industrial IoT platform for front-end distributed services according to claim 1, characterized in that, The mapping module is further configured to: obtain a mapping matrix, wherein the mapping matrix represents the mapping relationship between any two of the three-dimensional image data, the first camera, and the second camera, and the mapping matrix includes a rotation matrix and a translation matrix; Based on the mapping matrix, the calibration value is determined; Based on the calibration value, the judgment point is mapped to the first image data to form the first calibration point, and the judgment point is mapped to the second image data to form the second calibration point; The mapping module is also configured to: Based on the mapping matrix, determine the predicted value of the mapping matrix; Based on the predicted values, the calibration values ​​of the mapping matrix are determined.

4. The intelligent manufacturing industrial IoT platform for front-end distributed services according to claim 3, characterized in that, The mapping matrix includes a first mapping matrix and a second mapping matrix, wherein the first mapping matrix is ​​a mapping between the three-dimensional image data and the first camera, and the second mapping matrix is ​​a mapping between the three-dimensional image data and the second camera. The mapping module is further configured to: predict a first predicted value of the first mapping matrix based on the first mapping matrix using a first prediction model, wherein the first prediction model is a machine learning model, and the input of the first prediction model includes a first theoretical value of the first mapping matrix, the conveyor belt speed, and motion sensor data. Based on the second mapping matrix, a second predicted value of the second mapping matrix is ​​predicted by a second prediction model, wherein the second prediction model is a machine learning model, and the input of the second prediction model includes the second theoretical value of the second mapping matrix, the conveyor belt speed, and the motion sensor data; The calibration values ​​include a first calibration value and a second calibration value. The mapping module is further configured to: determine the first calibration value of the first mapping matrix and the second calibration value of the second mapping matrix based on the first predicted value and the second predicted value using a binocular calibration model. The binocular calibration model is a machine learning model. The first calibration value includes a first rotation matrix calibration value and a first translation matrix calibration value. The second calibration value includes a second rotation matrix calibration value and a second translation matrix calibration value. The first calibration value is used to map the judgment point to the first image data to form the first calibration point. The second calibration value is used to map the judgment point to the second image data to form the second calibration point. The training methods for the first prediction model, the second prediction model, and the stereo calibration model include joint training.

5. The intelligent manufacturing industrial IoT platform for front-end distributed services according to claim 4, characterized in that, The input of the binocular calibration model includes reference point information, wherein the reference point information includes at least one of a first position and a second position of the reference point, wherein the reference point is a preset point in the three-dimensional image data, the first position is the position of the reference point on the first image data, and the second position is the position of the reference point on the second image data. The reference point information also includes an offset, wherein the offset is the vector difference between the first position vector and the second position vector of the second position, wherein the first position vector is the position vector of the first position mapped onto the second image data based on the binocular prediction value of the first position based on the binocular mapping matrix, and the binocular mapping matrix is ​​the mapping between the first camera and the second camera; The mapping module is also configured to: The binocular prediction value is determined based on the first prediction value and the second prediction value.

6. A front-end service platform intelligent manufacturing industrial IoT control method, applied to a service platform, management platform, and sensor network platform that interact sequentially, characterized in that, The service platform includes a main service platform and multiple sub-service platforms; The control method executed by the management platform includes: The sensor network platform acquires images captured by the first camera on the production line as first image data, and images captured by the second camera as second image data; the first camera and the second camera form a binocular camera. The first image data and the second image data are processed into three-dimensional image data; When distortion of the three-dimensional image data is detected, multiple points of the distorted area are obtained as judgment points; The judgment points are mapped onto the first image data to form first calibration points, and the judgment points are mapped onto the second image data to form second calibration points; The first camera and the second camera are calibrated based on the first calibration point and the second calibration point; The service sub-platforms respectively receive calibration data from different binocular cameras; The service platform aggregates all the calibration data and displays it through the user platform; When distortion of the three-dimensional image data is detected, obtaining multiple points of the distorted area as judgment points includes: detecting whether the three-dimensional image data is distorted according to a preset template; When distortion of the three-dimensional image data is detected, a region of interest is established centered on the point of maximum distortion. Based on the preset template, at least two consecutive distorted edges are identified from the region of interest. If no at least two consecutive distorted edges are identified, the region of interest is redefined and identified again. According to a preset template, separate edges and shared edges are identified from the distorted edges, and multiple points are extracted from the separate edges as first points and multiple points are extracted from the shared edges as second points; the separate edges are the edges where the same points in the first image data and the second image data do not overlap in the three-dimensional image data; the shared edges are the edges where the same points in the first image data and the second image data overlap in the three-dimensional image data. The first point and the second point are used as the judgment points.

7. The intelligent manufacturing industrial IoT control method for the front-end service platform according to claim 6, characterized in that, The calibration of the first camera and the second camera based on the first calibration point and the second calibration point includes: during the calibration of the first camera and the second camera based on the first calibration point and the second calibration point, inverting the calibration matrix that generates the three-dimensional image data based on the first verification point, and verifying the inversion result based on the second verification point to form the adjustment parameters of the first camera and / or the second camera; the first verification point is the first calibration point and the second calibration point generated by mapping the first point; the second verification point is the first calibration point and the second calibration point generated by mapping the second point; The first camera and / or the second camera are calibrated according to the adjustment parameters. The preset template is configured with multiple recognition edges; Detecting whether the three-dimensional image data is distorted according to a preset template includes: The preset template is periodically compared with the three-dimensional image data, and the three-dimensional image data is judged to be distorted when the difference between any recognized edge and the corresponding edge data in the three-dimensional image data is greater than a preset value. Identifying separate edges and shared edges from the distorted edges based on a preset template includes: Based on the recognition edge in the preset template, the distorted edge is identified from the region of interest as a separating edge, and the separating edge is extended along the recognition edge to form a shared edge.

8. A computer-readable storage medium, characterized in that, The storage medium stores computer instructions, which, when executed by a processor, implement the intelligent manufacturing industrial IoT control method of the front-end service platform as described in claim 6.

Images