A production system, method, device and Internet of Things based on digitalization

Abstract

The present application proposes a production system, method, device and Internet of Things based on digitalization, the core solution of which includes: a first observation device, a first robot system, a first camera and a second camera, the first camera can be rotatably installed on the first robot system, the second camera is fixedly installed on the first robot system, and the first camera and the second camera are aimed in different directions; the robot system also includes a configuration reading unit, a first control unit, a first image processing unit, a second control unit, a second image processing unit and a first transmission unit. The digital production system, method and device proposed in the present application can efficiently monitor and protect sensitive areas in the production environment in real time through advanced image processing technology and robot automation technology, and avoid leaks while conveniently observing the factory status.

Description

Technical Field

[0001] The present invention relates to the field of digital production, and in particular to a digital-based production system, method, device and Internet of Things. Background Art

[0002] With the rapid development of industrial automation and information technology, accurate identification and protection of sensitive areas in digital production environments is a major challenge. In particular, when it comes to inspections, visits or inspections of factory buildings or production experimental bases, existing technologies have limitations in real-time monitoring, image processing accuracy, and flexibility in authority management. Especially when dealing with the protection of sensitive information in highly dynamic environments, how to accurately identify and process sensitive areas in real time during inspections and visits to production experimental bases remains a problem that needs to be solved. Therefore, there is a need for an efficient system and method that can intelligently identify sensitive areas and perform corresponding processing based on authority. Summary of the invention

[0003] The present application proposes a digital-based production system, method, device and Internet of Things, which can efficiently monitor and protect sensitive areas in the production environment in real time through advanced image processing technology and robotic automation technology.

[0004] On the one hand, the present application proposes a production system based on digitalization, comprising a first observation device, a first robot system, a first camera and a second camera, wherein the first camera can be rotatably installed on the first robot system, and the second camera is fixedly installed on the first robot system, and the first camera and the second camera are aligned in different directions; the robot system further comprises a configuration reading unit, a first control unit, a first image processing unit, a second control unit, a second image processing unit and a first transmission unit, characterized in that:

[0005] The configuration reading unit is used to read the delay configuration information of the first observation device, wherein the delay configuration information indicates the time difference between the image captured by the first robot system and the image displayed by the first observation device;

[0006] The first control unit is used to drive the first camera to rotate to a preset angle;

[0007] The first camera is used to capture a first image;

[0008] The first image processing unit is used to identify a first sensitive area from the first image according to a first recognition model;

[0009] The second control unit is used to control the robot system to rotate, thereby driving the second camera to rotate;

[0010] The second camera is used to capture a second image, wherein the second image has a higher definition than the first image;

[0011] The second image processing unit is used to determine the first to-be-recognized area of ​​the second image based on the first sensitive area and the delay configuration information; identify the second sensitive area from the first to-be-recognized area according to the second recognition model; replace the image at the second sensitive area with a preset image to obtain a target image;

[0012] The first transmission unit is used to send the target image to the first observation device;

[0013] The first observation device is used to output the target image to a display device.

[0014] The digital production system has the following advantages: Since the first camera can rotate relative to the robot (for example, move in a circle around the waist of the robot), the surrounding scene can be photographed in advance. The first image processing unit preliminarily recognizes the first image with low clarity shot by the first camera in advance and determines the first sensitive area. When the second camera (fixed on the body of the robot) follows the body of the robot to the shooting angle of the first image or a position close to the shooting angle of the first image, the second image processing unit begins to enlarge the size of the first sensitive area according to the observation time difference indicated by the delay configuration information. For example, when the allowed time difference is large, the second image processing unit can have more time to process the area to be identified in the second image, and the first sensitive area can be enlarged by a larger multiple, thereby determining the size of the area to be identified in the second image. Among them, the area to be identified in the second image is smaller than the overall size of the second image. After determining the area to be identified in the second image, the second image processing unit can process the area to be identified, identify sensitive areas that are not suitable for display to users, and replace or block them to avoid leakage. The digital production system proposed in the present application determines the area to be identified in the second image taken by the second camera through the preprocessing function of the first camera, and then identifies the sensitive area by processing the area to be identified. Since the size of the sensitive area is much smaller than the overall size of the second image, the recognition time can be greatly shortened and the display delay can be reduced.

[0015] Through the technical solution provided in this application, the production base can be monitored and protected by using digital technology, which greatly improves the real-time monitoring of the production base, image processing accuracy, and flexibility of authority management.

[0016] In a possible implementation, the first observation device is also used to: receive permission information input by a user, and verify the permission information; replace the image at the second sensitive area with a preset image to obtain a target image, including: when it is determined that the permission information verification fails, replace the image at the second sensitive area with a preset image to obtain a target image.

[0017] This implementation plan first verifies the user's authority. If the verification fails, it means that the user does not have sufficient authority to view all contents inside the factory. Only then should the internal sensitive information be blocked.

[0018] Once the system identifies sensitive areas, it will decide on the processing strategy for these areas based on pre-set user permission rules. For example, for unauthorized observers, the system may replace or blur the image of sensitive areas to prevent the leakage of confidential information. This permission-based image processing not only improves the security of the system, but also provides the necessary flexibility for operators.

[0019] In a possible implementation, the determining of the first area to be identified in the second image based on the first sensitive area and the delay configuration information includes: identifying the size of the first sensitive area when the shooting angle difference between the first image and the second image is less than a first angle threshold; determining the time difference indicated by the delay configuration information, determining the magnification ratio according to the time difference indicated by the delay configuration information, and magnifying the size of the first sensitive area according to the magnification ratio to obtain the size of the first area to be identified. The larger the time difference, the larger the magnification ratio.

[0020] In a possible implementation, the above-mentioned determination of the first area to be identified in the second image based on the first sensitive area and the delay configuration information also includes: when the shooting angle difference between the first image and the second image is greater than the first angle threshold, determining the homography transformation matrix between the first image and the second image; determining the first area to be determined based on the first sensitive area and the homography transformation matrix; determining the time difference indicated by the delay configuration information, determining the magnification ratio based on the time difference indicated by the delay configuration information, and enlarging the size of the first area to be determined according to the magnification ratio to obtain the size of the first area to be identified.

[0021] This application configures the time difference by setting a delay, that is, the delay between the photo / video taken by the camera and the video displayed on the display. The main reason for the delay is that it takes time to process sensitive image information. If the user sets a long delay, there is more time to process sensitive image areas. At this time, a larger area can be identified to avoid missing the detection area.

[0022] In this embodiment, when the alignment directions of the first camera and the second camera are different, the shooting angle difference between the first image and the second image is greater than the first angle threshold. At this time, it is necessary to perform a homography transformation on the first sensitive area to facilitate the second image processor to determine the area to be identified in the second image based on the area where the image after the homography transformation is located.

[0023] In a possible implementation, the robot system further includes an angle determination unit for obtaining shooting angle information of the first image and the second image. After obtaining the shooting angle information, the angle difference between the first image and the second image can be determined based on the shooting angle information to facilitate image transformation.

[0024] Since the first camera is rotatably installed on the above-mentioned first robot system, and the above-mentioned second camera is fixedly installed on the above-mentioned first robot system, there is a difference in the real-time shooting angles of the first camera and the second camera. By adjusting the viewing angles of the first image and the second image to be consistent, the area to be identified in the second image can be determined more accurately, avoiding misjudgment due to inconsistent shooting angles.

[0025] In a possible implementation, the first recognition model includes a dynamically updated Scharr operator, and the step of identifying the first sensitive area from the first image according to the first recognition model includes:

[0026] Convolution operation is performed on the first image and the above-mentioned dynamically updated Scharr operator to obtain a horizontal output feature map Ox and a vertical output feature map Oy; wherein Ox=I*Gx(θx), Oy=I*Gy(θy), * represents a convolution operation, I is the above-mentioned first image, θx is a horizontal convolution kernel parameter, θy is a vertical convolution kernel parameter, Gx is a horizontal convolution kernel of the Scharr operator, and Gy is a vertical convolution kernel Gy of the Scharr operator;

[0027] The first edge intensity image M(i, j) is calculated based on the horizontal output feature map Ox and the vertical output feature map Oy. The calculation formula is:

[0028]

[0029] The first edge intensity image M(i, j) is binarized to obtain a first binarized image, and an area with a pixel value of 1 in the first binarized image is used as the first sensitive area.

[0030] In this implementation, dynamically updating the Scharr operator can improve the processing efficiency of the low-resolution image and identify the first sensitive area as quickly as possible.

[0031] In a possible implementation, before performing a convolution operation on the first image and the dynamically updated Scharr operator, the method further includes:

[0032] The above horizontal convolution kernel parameters θx and vertical convolution kernel parameters θy are updated according to the following formula:

[0033]

[0034]

[0035] Among them, θx1 and θy1 are the dynamically updated horizontal and vertical convolution kernel parameters, a is the learning rate, and Respectively represent the loss function J(θ x ) and J(θ y ) relative to the parameter θ x and θ y gradient.

[0036] In this embodiment, the dynamically updated parameters θx1 and θy1 are added to the two convolution kernels, and then the first image is convolved with the dynamically updated Scharr operator to obtain a horizontal output feature map Ox and a vertical output feature map Oy. Updating the convolution kernel parameters according to gradient descent can improve the accuracy of recognition and further reduce the possibility of leakage.

[0037] In a possible implementation, the identifying the second sensitive area from the first area to be identified based on the second recognition model includes: scanning the image input of the first area to be identified using a Canny edge detection model to obtain the second sensitive area.

[0038] After determining the first area to be identified, since the size of the image of the first area to be identified is much smaller than the second image and the image clarity is higher, the more accurate Canny edge detection model can be used to identify the first area to be identified, thereby accurately obtaining the second sensitive area.

[0039] The image processing unit in the system of the present application can efficiently identify sensitive areas in a complex factory environment through the dynamically updated Scharr operator model and Canny edge detection model. The Scharr operator model is more sensitive when capturing image edges, which helps to depict clear regional boundaries, while the Canny edge detection provides precise edge recognition to ensure accurate detection of sensitive areas.

[0040] In one possible implementation, the image at the second sensitive area is replaced with a preset image to obtain a target image, including: acquiring the preset image from a portion of the second image other than the second sensitive area, and replacing the image at the second sensitive area with the preset image to obtain the target image.

[0041] On the other hand, the present application proposes a digital production method for a digital production system, wherein the digital production system comprises a first observation device, a first robot system, a first camera and a second camera, wherein the first camera can be rotatably mounted on the first robot system, and the second camera is fixedly mounted on the first robot system, and the first camera and the second camera are aligned in different directions; the robot system further comprises a configuration reading unit, a first control unit, a first image processing unit, a second control unit, a second image processing unit and a first transmission unit, wherein the digital production method comprises:

[0042] Reading the delay configuration information of the first observation device through a configuration reading unit, wherein the delay configuration information indicates a time difference between the first robot system capturing an image and the first observation device displaying the image;

[0043] The first control unit drives the first camera to rotate to a preset angle;

[0044] Capturing a first image through the first camera;

[0045] Identifying a first sensitive area from the first image according to a first recognition model by the first image processing unit;

[0046] Control the robot system to rotate by the second control unit, thereby driving the second camera to rotate;

[0047] capturing a second image through the second camera, wherein the second image has a higher definition than the first image;

[0048] Determine the first to-be-recognized area of ​​the second image based on the first sensitive area and the delay configuration information by the second image processing unit; identify the second sensitive area from the first to-be-recognized area according to the second recognition model; replace the image at the second sensitive area with a preset image to obtain a target image;

[0049] Sending the target image to the first observation device through the first transmission unit;

[0050] The target image is output to a display device through the first observation device.

[0051] On the other hand, the present application proposes a digital-based production device, comprising at least one processor, such as a central processing unit (CPU), at least one memory, and at least one bus. The memory may store program instructions, and the processor may be used to call the program instructions to execute a digital-based production method.

[0052] On the other hand, the present application also proposes a digital-based production Internet of Things, which includes the aforementioned digital-based production system and network equipment, and the network equipment includes routers, switches, base stations, etc. The aforementioned digital-based production Internet of Things can efficiently realize the aforementioned digital production functions.

[0053] Using the solution of the present application, the system uses the first and second cameras to capture images, and accurately identifies and processes sensitive areas in the images through the first and second image processing units combined with the dynamic Scharr operator model and the Canny edge detection model. In addition, the system also includes an observation device for authority management, which can decide whether to replace the image of the sensitive area according to the user's authority, so as to protect sensitive information. This system strengthens the factory's security measures, improves operational efficiency, and reduces the risks caused by misoperation or unauthorized access. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the background technology, the drawings required for use in the embodiments of the present application or the background technology will be described below.

[0055] Figure 1 It is a digital-based production system proposed in this application;

[0056] Figure 2 It is a digital-based production method proposed in this application;

[0057] Figure 3 It is a digital production device proposed in this application. DETAILED DESCRIPTION

[0058] The terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods or devices.

[0059] Figure 1It is a production system based on digitalization proposed in this application.

[0060] The above-mentioned digital-based production system includes a first observation device 109, a first robot system 100, a first camera 103 and a second camera 106. The above-mentioned first camera can be rotatably installed on the above-mentioned first robot system 100, and the above-mentioned second camera 106 is fixedly installed on the above-mentioned first robot system 100. The directions in which the above-mentioned first camera 103 and the above-mentioned second camera 106 are aimed are different; the above-mentioned robot system also includes a configuration reading unit 101, a first control unit 102, a first image processing unit 104, a second control unit 105, a second image processing unit 107 and a first transmission unit 108.

[0061] The configuration reading unit 101 is used to read the delay configuration information of the first observation device, where the delay configuration information indicates the time difference between the first robot system capturing the image and the first observation device displaying the image;

[0062] The first control unit 102 is used to drive the first camera to rotate to a preset angle;

[0063] The first camera 103 is used to capture a first image;

[0064] The first image processing unit 104 is configured to identify a first sensitive area from the first image according to a first recognition model;

[0065] The second control unit 105 is used to control the robot system to rotate, thereby driving the second camera to rotate;

[0066] The second camera 106 is used to capture a second image, wherein the second image has a higher definition than the first image;

[0067] The second image processing unit 107 is used to determine the first to-be-recognized area of ​​the second image based on the first sensitive area and the delay configuration information; identify the second sensitive area from the first to-be-recognized area according to the second recognition model; replace the image at the second sensitive area with a preset image to obtain a target image;

[0068] The first transmission unit 108 is used to send the target image to the first observation device;

[0069] The first observation device 109 is used to output the target image to a display device.

[0070] The first camera 103 is rotatably mounted on the first robot system, which means that the first camera 103 can move around the robot body to capture images of multiple angles around the robot body. The second camera 106 is fixedly mounted on the first robot system and cannot move relative to the robot body, that is, it can only change its aiming direction as the body rotates.

[0071] The delay configuration information indicates the time difference between the first robot system capturing an image and the first observation device 109 displaying the image, that is, the display delay allowed by the user, which can be adjusted according to the user's setting instructions.

[0072] The first camera 103 is a low-definition camera, and the second camera 106 is a high-definition camera. The clarity or resolution of the picture taken by the first camera 103 is low, which can reduce the memory and the computing power required for image processing to a certain extent. The clarity or resolution of the picture taken by the second camera 106 is high, which can improve the accuracy of sensitive area recognition.

[0073] The first observation device 109 may be a VR device, and may input a command for setting a display delay. After receiving the image information sent by the robot system, the observation device outputs it to a display screen or VR glasses and other devices, so that the user can observe the environment in the factory.

[0074] The size of the first area to be identified in the second image is much smaller than the overall size of the second image. The second image processor only identifies the first area to be identified in the second image, which can greatly reduce computing power loss and improve computing efficiency.

[0075] The digital production system has the following advantages: Since the first camera can rotate relative to the robot (for example, move in a circle around the waist of the robot), the surrounding scene can be photographed in advance. The first image processing unit preliminarily recognizes the first image with low clarity shot by the first camera in advance and determines the first sensitive area. When the second camera (fixed on the body of the robot) follows the body of the robot to the shooting angle of the first image or a position close to the shooting angle of the first image, the second image processing unit begins to enlarge the size of the first sensitive area according to the observation time difference indicated by the delay configuration information. For example, when the allowed time difference is large, the second image processing unit can have more time to process the area to be identified in the second image, and the first sensitive area can be enlarged by a larger multiple, thereby determining the size of the area to be identified in the second image. Among them, the area to be identified in the second image is smaller than the overall size of the second image. After determining the area to be identified in the second image, the second image processing unit can process the area to be identified, identify sensitive areas that are not suitable for display to users, and replace or block them to avoid leakage. The digital production system proposed in the present application determines the area to be identified in the second image taken by the second camera through the preprocessing function of the first camera, and then identifies the sensitive area by processing the area to be identified. Since the size of the sensitive area is much smaller than the overall size of the second image, the recognition time can be greatly shortened and the display delay can be reduced.

[0076] Through the technical solution provided in this application, the production system can be monitored and protected by digital technology, which greatly improves the real-time monitoring of the production system, image processing accuracy, and flexibility of authority management.

[0077] In a possible implementation, the first observation device 109 is also used to: receive permission information input by a user, and verify the permission information; replace the image at the second sensitive area with a preset image to obtain a target image, including: when it is determined that the permission information verification fails, replace the image at the second sensitive area with a preset image to obtain a target image.

[0078] This implementation plan first verifies the user's authority. If the verification fails, it means that the user does not have sufficient authority to view all contents inside the factory. Only then should the internal sensitive information be blocked.

[0079] Once the system identifies sensitive areas, it will decide on the processing strategy for these areas based on pre-set user permission rules. For example, for unauthorized observers, the system may replace or blur the image of sensitive areas to prevent the leakage of confidential information. This permission-based image processing not only improves the security of the system, but also provides the necessary flexibility for operators.

[0080] In a possible implementation, the determining of the first area to be identified in the second image based on the first sensitive area and the delay configuration information includes: identifying the size of the first sensitive area when the shooting angle difference between the first image and the second image is less than a first angle threshold; determining the time difference indicated by the delay configuration information, determining the magnification ratio according to the time difference indicated by the delay configuration information, and magnifying the size of the first sensitive area according to the magnification ratio to obtain the size of the first area to be identified. The larger the time difference, the larger the magnification ratio.

[0081] If the shooting angle difference between the first image and the second image is less than the first angle threshold, it means that the first camera and the second camera are pointed in substantially the same direction, and the pictures taken by the two cameras are almost the same.

[0082] In a possible implementation, in order to make the images captured by the two cameras as consistent as possible, the first camera and the second camera may be placed at the same height.

[0083] In a possible implementation, the above-mentioned determination of the first area to be identified in the second image based on the first sensitive area and the delay configuration information also includes: when the shooting angle difference between the first image and the second image is greater than the first angle threshold, determining the homography transformation matrix between the first image and the second image; determining the first area to be determined based on the first sensitive area and the homography transformation matrix; determining the time difference indicated by the delay configuration information, determining the magnification ratio based on the time difference indicated by the delay configuration information, and enlarging the size of the first area to be determined according to the magnification ratio to obtain the size of the first area to be identified.

[0084] When the shooting angle difference between the first image and the second image is greater than the first angle threshold, it proves that the directions of the two cameras are greatly different, and the overlap between the first image and the second image is relatively small.

[0085] This application configures the time difference by setting a delay, that is, the delay between the photo / video taken by the camera and the video displayed on the display. The main reason for the delay is that it takes time to process sensitive image information. If the user sets a long delay, there is more time to process sensitive image areas. At this time, a larger area can be identified to avoid missing the detection area.

[0086] In this embodiment, when the alignment directions of the first camera and the second camera are different, the shooting angle difference between the first image and the second image is greater than the first angle threshold. At this time, it is necessary to perform a homography transformation on the first sensitive area to facilitate the second image processor to determine the area to be identified in the second image based on the area where the image after the homography transformation is located.

[0087] In a possible implementation, the robot system further includes an angle determination unit for obtaining shooting angle information of the first image and the second image. After obtaining the shooting angle information, the angle difference between the first image and the second image can be determined based on the shooting angle information to facilitate image transformation.

[0088] Since the first camera is rotatably installed on the above-mentioned first robot system, and the above-mentioned second camera is fixedly installed on the above-mentioned first robot system, there is a difference in the real-time shooting angles of the first camera and the second camera. The viewing angles of the first image and the second image are adjusted to be consistent through homography transformation, so that the area to be identified in the second image can be determined more accurately, avoiding misjudgment due to inconsistent shooting angles.

[0089] In a possible implementation, the first recognition model includes a dynamically updated Scharr operator, and the step of identifying the first sensitive area from the first image according to the first recognition model includes:

[0090] Convolution operation is performed on the first image and the above-mentioned dynamically updated Scharr operator to obtain a horizontal output feature map Ox and a vertical output feature map Oy; wherein Ox=I*Gx(θx), Oy=I*Gy(θy), * represents a convolution operation, I is the above-mentioned first image, θx is a horizontal convolution kernel parameter, θy is a vertical convolution kernel parameter, Gx is a horizontal convolution kernel of the Scharr operator, and Gy is a vertical convolution kernel Gy of the Scharr operator;

[0091] The first edge intensity image M(i, j) is calculated based on the horizontal output feature map Ox and the vertical output feature map Oy. The calculation formula is:

[0092]

[0093] The first edge intensity image M(i, j) is binarized to obtain a first binarized image, and an area with a pixel value of 1 in the first binarized image is used as the first sensitive area.

[0094] In this implementation, dynamically updating the Scharr operator can improve the processing efficiency of the low-resolution image and identify the first sensitive area as quickly as possible.

[0095] In a possible implementation, before performing a convolution operation on the first image and the dynamically updated Scharr operator, the method further includes:

[0096] The above horizontal convolution kernel parameters θx and vertical convolution kernel parameters θy are updated according to the following formula:

[0097]

[0098]

[0099] Among them, a is the learning rate, and Respectively represent the loss function J(θ x ) and J(θ y ) relative to the parameter θ x and θ y gradient.

[0100] In this embodiment, the dynamically updated parameters θx1 and θy1 are added to the two convolution kernels, and then the first image is convolved with the dynamically updated Scharr operator to obtain a horizontal output feature map Ox and a vertical output feature map Oy. Updating the convolution kernel parameters according to gradient descent can improve the accuracy of recognition and further reduce the possibility of leakage.

[0101] In a possible implementation, the identifying the second sensitive area from the first area to be identified based on the second recognition model includes: scanning the image input of the first area to be identified using a Canny edge detection model to obtain the second sensitive area.

[0102] After determining the first area to be identified in the second image, since the size of the first area to be identified is much smaller than the second image and the image clarity is higher, the more accurate Canny edge detection model can be used to identify the first area to be identified, thereby accurately obtaining the second sensitive area.

[0103] The image processing unit in the system of the present application can efficiently identify sensitive areas in a complex factory environment through the dynamically updated Scharr operator model and Canny edge detection model. The Scharr operator model is more sensitive when capturing image edges, which helps to depict clear regional boundaries, while the Canny edge detection provides precise edge recognition to ensure accurate detection of sensitive areas.

[0104] In one possible implementation, the image at the second sensitive area is replaced with a preset image to obtain a target image, including: acquiring the preset image from a portion of the second image other than the second sensitive area, and replacing the image at the second sensitive area with the preset image to obtain the target image.

[0105] In addition, a preset image, such as a blue-background image, a red-background image, or a cartoon image, may be read to replace the image at the second sensitive area with the preset image.

[0106] Figure 2 The present application proposes a digital production method for a digital production system, wherein the digital production system comprises a first observation device, a first robot system, a first camera and a second camera, wherein the first camera can be rotatably mounted on the first robot system, and the second camera is fixedly mounted on the first robot system, and the first camera and the second camera are aligned in different directions; the robot system further comprises a configuration reading unit, a first control unit, a first image processing unit, a second control unit, a second image processing unit and a first transmission unit, and is characterized in that the digital production method comprises:

[0107] 201. Reading delay configuration information of the first observation device through a configuration reading unit, wherein the delay configuration information indicates a time difference between an image captured by the first robot system and an image displayed by the first observation device;

[0108] 202. Drive the first camera to rotate to a preset angle by the first control unit;

[0109] 203. Capturing a first image through the first camera;

[0110] 204. Identify a first sensitive area from the first image according to a first recognition model by the first image processing unit;

[0111] 205. Control the robot system to rotate by the second control unit, thereby driving the second camera to rotate;

[0112] 206. Capturing a second image through the second camera, wherein the second image has a higher definition than the first image;

[0113] 207. Determine, by the second image processing unit, a first to-be-recognized area of ​​the second image based on the first sensitive area and the delay configuration information; identify a second sensitive area from the first to-be-recognized area according to a second recognition model; replace the image at the second sensitive area with a preset image to obtain a target image;

[0114] 208. Send the target image to a first observation device through the first transmission unit;

[0115] 209. Output the target image to a display device through the first observation device.

[0116] For the extended parts and details of the aforementioned digital-based production method, please refer to the implementation example of the distributed power supply system based on the smart energy management platform, which will not be repeated here.

[0117] like Figure 3 The figure is a schematic diagram of a digital production device provided by the present application. The device includes: at least one processor 310, such as a central processing unit (CPU), at least one memory 320, and at least one bus 330. The memory 320 may store program instructions, and the processor 310 may be used to call the program instructions to execute a digital production method.

[0118] On the other hand, the present application also proposes a digital-based production Internet of Things, which includes the aforementioned digital-based production system and network equipment, and the network equipment includes routers, switches, base stations, etc. The aforementioned digital-based production Internet of Things can efficiently realize the aforementioned monitoring and identification functions for the production base or factory.

Claims

1. A production system based on digitalization, comprising a first observation device, a first robot system, a first camera and a second camera, wherein the first camera can be rotatably mounted on the first robot system, and the second camera is fixedly mounted on the first robot system, and the first camera and the second camera are aligned in different directions; the first robot system further comprises a configuration reading unit, a first control unit, a first image processing unit, a second control unit, a second image processing unit and a first transmission unit, characterized in that: The configuration reading unit is used to read the delay configuration information of the first observation device, wherein the delay configuration information indicates the time difference between the first robot system capturing the image and the first observation device displaying the image; The first control unit is used to drive the first camera to rotate at a preset angle; The first camera is used to capture a first image; The first image processing unit is used to identify a first sensitive area from the first image according to a first recognition model; The second control unit is used to control the first robot system to rotate, thereby driving the second camera to rotate; The second camera is used to capture a second image, wherein the second image has a higher definition than the first image; The second image processing unit is used to determine a first to-be-identified area of ​​the second image based on the first sensitive area and the delay configuration information; identifying a second sensitive area from the first to-be-identified area according to a second identification model; Replacing the image at the second sensitive area with a preset image to obtain a target image; The first transmission unit is used to send the target image to a first observation device; The first observation device is used to output the target image to a display device; The determining the first to-be-identified area of ​​the second image based on the first sensitive area and the delay configuration information includes: When the shooting angle difference between the first image and the second image is less than a first angle threshold, identifying a size of the first sensitive area; Determine the time difference indicated by the delay configuration information, determine the magnification ratio according to the time difference indicated by the delay configuration information, and magnify the size of the first sensitive area according to the magnification ratio to obtain the size of the first area to be identified; The difference in shooting angles between the first image and the second image is based on the different directions in which the first camera and the second camera are pointed.

2. The production system according to claim 1, characterized in that: The first observation device is also used for: Receiving permission information input by a user, and verifying the permission information; Replacing the image at the second sensitive area with a preset image to obtain a target image includes: When it is determined that the permission information verification fails, the image at the second sensitive area is replaced with a preset image to obtain a target image.

3. The production system according to claim 2, characterized in that: The determining the first to-be-identified area of ​​the second image based on the first sensitive area and the delay configuration information further includes: When the shooting angle difference between the first image and the second image is greater than a first angle threshold, determining a homography transformation matrix between the first image and the second image; Determining a first undetermined area according to the first sensitive area and the homography transformation matrix; Determine the time difference indicated by the delay configuration information, determine the magnification ratio according to the time difference indicated by the delay configuration information, and magnify the size of the first to-be-identified area according to the magnification ratio to obtain the size of the first to-be-identified area.

4. The production system according to claim 3, characterized in that: The first robot system further includes an angle judgment unit, which is used to obtain shooting angle information of the first image and the second image.

5. The production system according to claim 4, characterized in that: The first recognition model includes a dynamically updated Scharr operator, and identifying a first sensitive area from the first image according to the first recognition model includes: The first image is convolved with the dynamically updated Scharr operator to obtain a horizontal output feature map , vertical output feature map ;in, , , represents the convolution operation, I is the first image, is the horizontal convolution kernel parameter, is the vertical convolution kernel parameter, Gx is the horizontal convolution kernel of the Scharr operator, and Gy is the vertical convolution kernel Gy of the Scharr operator; Output feature map according to level , vertical output feature map Calculate the first edge strength image , the calculation formula is: For the first edge intensity image M A binarization process is performed to obtain a first binarized image, and an area in the first binarized image where the pixel value is 1 is used as the first sensitive area.

6. The production system according to claim 5, characterized in that: Before performing a convolution operation on the first image and the dynamically updated Scharr operator, the method further includes: For the horizontal convolution kernel parameters and vertical convolution kernel parameters Update according to the formula: in, and are the horizontal and vertical convolution kernel parameters after dynamic update, α is the learning rate, and Represent the loss function and Relative to the parameters and gradient.

7. The production system according to claim 5, characterized in that: The step of identifying a second sensitive area from the first area to be identified according to the second identification model includes: The image input of the first area to be identified is scanned using the Canny edge detection model to obtain the second sensitive area.

8. The production system according to claim 7, characterized in that: Replacing the image at the second sensitive area with a preset image to obtain a target image includes: The preset image is acquired from a portion of the second image excluding the second sensitive area, and the image at the second sensitive area is replaced with the preset image to obtain a target image.

9. A production method based on digitalization, used in a digital production system, the digital production system comprising a first observation device, a first robot system, a first camera and a second camera, the first camera can be rotatably installed on the first robot system, the second camera is fixedly installed on the first robot system, and the first camera and the second camera are aligned in different directions; the first robot system also includes a configuration reading unit, a first control unit, a first image processing unit, a second control unit, a second image processing unit and a first transmission unit, characterized in that: The digital production method comprises: Reading delay configuration information of the first observation device through a configuration reading unit, wherein the delay configuration information indicates a time difference between the first robot system capturing an image and the first observation device displaying the image; Driving the first camera to rotate to a preset angle by the first control unit; capturing a first image through the first camera; identifying a first sensitive area from the first image according to a first recognition model by the first image processing unit; Controlling the first robot system to rotate by the second control unit, thereby driving the second camera to rotate; capturing a second image through the second camera, wherein the second image has a higher definition than the first image; Determine, by the second image processing unit, a first area to be identified in the second image based on the first sensitive area and the delay configuration information; identify a second sensitive area from the first area to be identified according to a second recognition model; replace the image at the second sensitive area with a preset image to obtain a target image; Sending the target image to a first observation device through the first transmission unit; Outputting the target image to a display device through the first observation device; The determining the first to-be-identified area of ​​the second image based on the first sensitive area and the delay configuration information includes: When the shooting angle difference between the first image and the second image is less than a first angle threshold, identifying a size of the first sensitive area; Determine the time difference indicated by the delay configuration information, determine the magnification ratio according to the time difference indicated by the delay configuration information, and magnify the size of the first sensitive area according to the magnification ratio to obtain the size of the first area to be identified; The difference in shooting angles between the first image and the second image is based on the different directions in which the first camera and the second camera are pointed.

Images