Artificial intelligence-based image processing method, device, system and storage medium

Abstract

This application discloses an image processing method, apparatus, system, and storage medium based on artificial intelligence. The method includes: acquiring a calibration image from a calibration device, the calibration device including a pattern board with multiple patterns; extracting features from the calibration image to obtain a first feature map; inputting the first feature map into a bounding box regression network to output multiple regions of interest (ROIs) corresponding to the calibration device in the first feature map; generating a preset number of candidate boxes for each ROI based on preset prior boxes, each candidate box having the same size as the preset prior box, the preset number being determined according to the number of patterns; and inputting all candidate boxes and the first feature map into a center regression network to obtain the coordinates of the image center point of each pattern in the calibration image. This scheme can obtain clear imaging edge information, and even under poor imaging quality, it can obtain accurate candidate regions, significantly improving the detection accuracy of image recognition.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and more specifically to an image processing method, apparatus, system and storage medium based on artificial intelligence. Background Technology

[0002] With the development of intelligent technology, image acquisition equipment is widely used in various fields. For example, visible light cameras, infrared thermal imaging cameras, and dual-light cameras (visible light and infrared thermal imaging cameras combined) all require calibration of their intrinsic parameters to correct distortion due to varying degrees of lens distortion. For instance, dual-light cameras used for fire source location and automatic fire suppression require precise calibration of their intrinsic parameters. However, precise calibration of camera intrinsic parameters places high demands on the camera's imaging resolution. For example, infrared cameras, which sense the distribution of infrared radiation energy, produce images that are less clear than visible light images. This is mainly manifested in lower image resolution, blurred object edges, and higher image noise, especially since large-area temperature anomalies can lead to failure in image feature extraction. Therefore, common feature extraction algorithms often suffer from significant errors in extracting features from infrared images. Summary of the Invention

[0003] The purpose of this application is to provide an image processing method, apparatus, system, and storage medium based on artificial intelligence to solve the technical problems of blurred image edge information and poor image recognition accuracy in the prior art.

[0004] To achieve the above objectives, the first aspect of this application provides an image processing method based on artificial intelligence, the method comprising:

[0005] Acquire calibration images from the calibration device, which includes a pattern plate with multiple patterns;

[0006] Feature extraction is performed on the calibration image to obtain the processed first feature map;

[0007] The first feature map is input into the bounding box regression network, so that the bounding box regression network outputs multiple regions of interest in the first feature map that correspond to the calibration device.

[0008] Based on the preset prior bounding boxes, a preset number of candidate bounding boxes are generated for each region of interest. The size of each candidate bounding box is the same as the size of the preset prior bounding box, and the preset number is determined according to the number of multiple patterns.

[0009] All candidate boxes and the first feature map are input into the center regression network to obtain the coordinates of the center point of each pattern in the calibration image.

[0010] A second aspect of this application provides an image processing apparatus based on artificial intelligence, comprising:

[0011] The memory is configured to store instructions; and

[0012] The processor is configured to retrieve instructions from memory and, when executing instructions, to implement the aforementioned AI-based image processing method.

[0013] A third aspect of this application provides an image processing system based on artificial intelligence, comprising:

[0014] Image acquisition equipment, used to acquire calibration images from the calibration device;

[0015] The calibration device includes a pattern board with multiple patterns, wherein the color of the pattern on the pattern board in the calibration image is different from the color of other areas;

[0016] The aforementioned image processing device based on artificial intelligence.

[0017] In the embodiments of this application, the calibration device further includes a heating plate connected to the pattern plate, such that when the heating plate is heated, each pattern in the calibration image presents a clear contrast with the background. The image acquisition device includes an infrared camera and a visible light camera, and the multiple patterns are multiple identical hollow patterns arranged in an array.

[0018] A fourth aspect of this application provides a machine-readable storage medium storing instructions that cause a machine to perform the above-described artificial intelligence-based image processing method.

[0019] The above technical solution involves designing a pattern board with multiple patterns on the calibration device's pattern plate. Calibration images are acquired for the calibration device, and feature extraction is performed on these images to obtain a first feature map. This transforms the image data in the calibration image into a form that is easy to process and analyze. A deep learning architecture combining a bounding box regression network and a center regression network is constructed. This deep learning architecture extracts multiple regions of interest from the first feature map. Combined with pre-designed prior boxes, each candidate region of interest is output to obtain the coordinates of the image center point of each pattern in the calibration image. This method can obtain clear imaging edge information and, even when the calibration image has blurred edges or poor imaging quality, obtains more accurate candidate regions, significantly improving the detection accuracy of image recognition.

[0020] Other features and advantages of the embodiments of this application will be described in detail in the following detailed description section. Attached Figure Description

[0021] The accompanying drawings are provided to further illustrate the embodiments of this application and form part of the specification. They are used together with the following detailed description to explain the embodiments of this application, but do not constitute a limitation on the embodiments of this application. In the drawings:

[0022] Figure 1 The illustration shows a flowchart of an artificial intelligence-based image processing method according to an embodiment of this application;

[0023] Figure 2 The illustration shows a schematic diagram of the process for generating a first feature map according to an embodiment of this application;

[0024] Figure 3 A schematic diagram illustrating the network structures of the center regression network and the border regression network according to embodiments of this application is shown.

[0025] Figure 4 The illustration shows a flowchart of an artificial intelligence-based image processing method according to a specific embodiment of this application;

[0026] Figure 5 A schematic flowchart of a PRPN network algorithm according to a specific embodiment of this application is shown.

[0027] Figure 6 This schematic diagram illustrates a structural block diagram of an artificial intelligence-based image processing system according to an embodiment of this application;

[0028] Figure 7 A schematic diagram of the calibration device according to an embodiment of this application is shown.

[0029] Figure 8 The illustration shows a schematic diagram of the process for determining camera intrinsic parameters according to an embodiment of this application;

[0030] Figure 9 This schematic diagram illustrates a structural block diagram of an artificial intelligence-based image processing apparatus according to an embodiment of this application;

[0031] Figure 10 The illustration shows a schematic diagram of the structure of a computer device according to an embodiment of the present application.

[0032] Explanation of reference numerals in the attached figures

[0033] 10 heating plates 20 pattern plates

[0034] 30. Hollow-out pattern Detailed Implementation

[0035] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only for illustration and explanation of the embodiments of this application and are not intended to limit the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0036] It should be noted that if the embodiments of this application involve directional indicators (such as up, down, left, right, front, back, etc.), the directional indicators are only used to explain the relative positional relationship and movement of the components in a certain specific posture (as shown in the figure). If the specific posture changes, the directional indicators will also change accordingly.

[0037] Furthermore, if the embodiments of this application involve descriptions such as "first" or "second," these descriptions are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, features defined with "first" or "second" may explicitly or implicitly include at least one of those features. Additionally, the technical solutions of various embodiments can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. If the combination of technical solutions is contradictory or impossible to implement, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed in this application.

[0038] Figure 1 The illustration shows a schematic flowchart of an image processing method according to an embodiment of this application. Figure 1 As shown in the figure, this application provides an image processing method based on artificial intelligence, which may include the following steps.

[0039] S102, Acquire calibration images from the calibration device, which includes a pattern plate with multiple patterns.

[0040] It is understood that a calibration device refers to a device used to determine and correct the measurement error of a sensor or measurement system. In the embodiments of this application, the calibration device is a device used to calibrate an image acquisition device. The calibration device includes a pattern plate with multiple patterns. The patterns on the pattern plate differ from other areas in shape, color, reflectivity, or temperature. Specifically, the image acquisition device can be an optical camera, an infrared camera, a scanner, etc. It is understood that a calibration image refers to an image acquired by the image acquisition device for the calibration device. If the image acquisition device is an optical camera, the calibration image is an optical image; if the image acquisition device is an infrared camera, the calibration image is an infrared image. The calibration of the optical image is based on the difference in shape and color between the pattern and other areas, while the calibration of the infrared image is based on the difference in temperature between the pattern and other areas. The pattern on the pattern board has a different color, shape, color, reflectivity, and temperature than other areas. This allows the pattern on the pattern board to be distinguished from other areas in the calibration image, resulting in clear imaging edge information. This facilitates the processing of the acquired calibration image, effectively completing the calibration of the image acquisition equipment and improving the accuracy and efficiency of the calibration.

[0041] S104, perform feature extraction on the calibration image to obtain the processed first feature map.

[0042] Image feature extraction refers to extracting representative features from raw image data for subsequent image analysis and processing tasks. In the embodiments of this application, the calibration image contains information about the pattern on the pattern board. By extracting features from the calibration image, a processed first feature map is obtained. The first feature map includes key attribute information such as color and shape from the calibration image.

[0043] In embodiments of this application, processing the calibration image to obtain a processed first feature map includes: inputting the calibration image into a residual network and outputting multiple second feature maps of different scales through the residual network; inputting the multiple second feature maps into an upsampling module and outputting multiple third feature maps of the same size through the upsampling module; concatenating all the third feature maps to obtain a fourth feature map; inputting the fourth feature map into an attention mechanism module and outputting a first feature map after pooling the fourth feature map in both spatial and channel dimensions through the attention mechanism module.

[0044] As can be understood, a residual network is a type of convolutional neural network. Each layer in a residual network learns the residual between the input and output; that is, the network's learning goal is to map the input to the desired output, thus optimizing network parameters and reducing training difficulty. The calibration image is input into the residual network, which outputs multiple second feature maps. The residual network can output corresponding second feature maps for different stages of the calibration image. These different second feature maps contain feature information at different scales of the calibration image. Further, these multiple second feature maps are input into an upsampling module, where each second feature map undergoes upsampling processing. In image processing, upsampling is a technique used to increase image resolution or recover missing information.

[0045] Specifically, upsampling methods include, but are not limited to, nearest neighbor interpolation, bilinear interpolation, bicubic interpolation, Lanczos interpolation, and the magnification factor method. In the embodiments of this application, after upsampling each second feature map, each second feature map is processed to the same size, so that multiple third feature maps of the same size are output through the upsampling module. Further, all third feature maps can be connected to link the features in the map together to obtain a fourth feature map. Specifically, all third feature maps can be connected by feature connection to obtain a fourth feature map. The fourth feature map includes the feature information of all third feature maps.

[0046] Furthermore, the fourth feature map is input into the attention mechanism module. This module includes channel attention (CAM) and spatial attention (SAM). The channel attention mechanism better understands different features in the image by calculating the importance of each channel, enabling the model to distinguish features between different channels. The spatial attention mechanism focuses on specific regions in the image, weighting the spatial importance of each pixel to better capture the spatial structure of the image. The fourth feature map can first undergo channel-dimensional pooling through the inter-channel attention mechanism, and then further undergo spatial-dimensional pooling through the spatial attention mechanism to obtain the processed first feature map. This effectively fuses features from different scales and channels of the image. The above scheme for extracting the first feature map can effectively remove image noise caused by external environmental factors.

[0047] Figure 2 The illustration shows a flowchart of generating a first feature map according to a specific embodiment of this application. Figure 2As shown, a ResNet50 residual network is used. The second feature map is output from each of ResNet50's stages 1, 2, 3, and 4. Each second feature map is then processed by an upsampling module (unpooling) to obtain four third feature maps of the same size: F1, F2, F3, and F4. These four identical feature maps are concatenated along the channel dimension using feature concatenation to obtain a fourth feature map, thus fusing four feature maps of different scales to fully utilize information from different scales. Furthermore, the fourth feature map is sequentially processed through a channel attention mechanism (CAM) and a spatial attention mechanism (SAM) to obtain a first feature map.

[0048] Specifically, channel attention mechanisms can better understand different features in an image. By calculating the importance of each channel, the model can distinguish features between different channels. The workflow of channel attention involves performing global max pooling and global average pooling on an input feature map of size H×W×C, outputting two 1×1×C feature maps. The results of global max pooling and global average pooling are then fed into a shared multilayer perceptron (MLP) to learn two more 1×1×C feature maps. The first layer of the MLP has C / r neurons, where r is the dimensionality reduction coefficient and the activation function is ReLU. The second layer has C neurons. The MLP learns the channel-dimensional features and the importance of each channel. Finally, the MLP output is added, and after mapping by the sigmoid activation function, the final channel attention weight matrix M is obtained. c Specifically, the functional expression of the channel attention mechanism is shown in the following formula (2):

[0049] M c (F a )=σ(MLP(AvgPool(F a ))+MLP(MaxPool(F a (2)

[0050] Among them, F a M refers to the feature map input to the channel attention mechanism. c (F a ) refers to the channel attention weight matrix, σ is the sigmoid activation function, MLP is the multilayer perceptron, AvgPool is the average pooling layer, and MaxPool is the max pooling layer.

[0051] The spatial attention module focuses on specific regions in the image, weighting the spatial importance of each pixel to better capture the spatial structure of the image. For an input feature map of size H×W×C, channel-wise global max pooling and global average pooling are performed, resulting in two feature maps of size H×W×1. The results of global max pooling and global average pooling are then concatenated by channel to obtain a feature map of size H×W×2. An m×m convolution operation is performed on the concatenated result to obtain a feature map of size H×W×1, which is then activated by a sigmoid function to obtain the spatial attention weight matrix M. s Specifically, the functional expression of the spatial attention mechanism is shown in the following formula (3):

[0052] M s (F b )=σ(f m×m ([AvgPool(F b MaxPool(F) b (3)

[0053] Among them, F b M refers to the feature map input to the spatial attention mechanism. s (F b ) refers to the spatial attention weight matrix, f m×m It is an m×m convolutional layer, σ is the sigmoid activation function, AvgPool is the average pooling layer, and MaxPool is the max pooling layer.

[0054] S106, the first feature map is input into the bounding box regression network so that the bounding box regression network outputs multiple regions of interest in the first feature map that correspond to the calibration device.

[0055] It is understandable that bounding box regression networks learn and predict correction values ​​from an initial object position (usually an initial bounding box) to enable the model to more accurately predict the object's bounding box. These correction values ​​include horizontal offset, vertical offset, width scaling, and height scaling, all relative to the initial bounding box. Inputting the first feature map into the bounding box regression network can fine-tune the initial bounding box, making it closer to the true location of the calibration device in the first feature map. That is, the bounding box regression network outputs multiple regions of interest (ROIs) in the first feature map corresponding to the calibration device. ROIs refer to regions in the first feature map with a high probability of containing the calibration device. In the embodiments of this application, identifying the feature map after removing image noise can effectively improve the recognition accuracy of ROIs.

[0056] S108, Based on the preset prior box, generate a preset number of candidate boxes for each region of interest. The size of each candidate box is the same as the size of the preset prior box, and the preset number is determined according to the number of multiple patterns.

[0057] Preset prior boxes refer to bounding boxes pre-defined for an image to be identified in object detection technology. The size of the preset prior boxes is pre-set. Specifically, the preset prior boxes can be set according to the size of the patterns on the calibration device. Understandably, the region of interest (ROI) is used to identify areas in the first feature map where the calibration device might exist, and candidate boxes are used to identify areas in the pattern board within the ROI where the patterns might exist. Based on the preset prior boxes, multiple candidate boxes can be generated for each ROI. The preset number refers to the pre-set number of candidate boxes to be generated for each ROI, which can be determined based on the number of patterns on the pattern board. For example, the preset number could be the number of patterns on the pattern board. By using preset prior boxes, a large number of candidate regions can be sampled, and it can be determined whether these regions contain the target, allowing for faster and more accurate prediction of the target's true bounding box.

[0058] In the embodiments of this application, multiple patterns are multiple identical and regularly arranged patterns. Based on a preset prior box, generating a preset number of candidate boxes for each region of interest includes: obtaining the size of the bounding rectangle of any pattern; generating a preset prior box based on the size of the bounding rectangle; and inputting the preset prior box and the region of interest to the region generation module for each region of interest, so that the region generation module outputs a preset number of candidate boxes for each region of interest.

[0059] It is understandable that, since the multiple patterns on the pattern board are all identical and arranged in a regular pattern, technicians can directly obtain the dimensions of the circumscribed rectangle of any pattern, i.e., the width and height of the circumscribed rectangle, when designing the calibration device. The dimensions of the circumscribed rectangle are scaled to generate a preset prior box that matches the first feature map. For each region of interest (ROI), the preset prior box and the ROI are input to the region generation module, which then outputs a preset number of candidate boxes for each ROI. The region generation module is the module that generates candidate boxes for each ROI using the preset prior box. By designing the preset prior box using the dimensions of the pattern's circumscribed rectangle, candidate boxes that are easy to identify can be generated quickly.

[0060] In the embodiments of this application, the function expression of the region generation module is shown in the following formula (1):

[0061]

[0062] Among them, Proposal (k，n) It refers to the nth candidate box in the kth region of interest, bbox.k It refers to the k-th region of interest, Base_anchor n It refers to the nth candidate box generated based on the preset prior box. x1 and x2 are the x-coordinates of the diagonal points of the bounding rectangle corresponding to the preset prior box, respectively. y1 and y2 are the y-coordinates of the diagonal points of the bounding rectangle corresponding to the preset prior box, respectively. x and y are the x-coordinates and y-coordinates of the fixed points of each region of interest, respectively. w and h are the width and height of each region of interest, respectively.

[0063] S110, input all candidate boxes and the first feature map into the center regression network to obtain the coordinates of the image center point of each pattern in the calibration image.

[0064] It is understandable that the center regression network is similar to the bounding box regression network, but the center regression network identifies the center point of the pattern on the pattern board within the candidate bounding box. The image center point of the pattern refers to the center point of the pattern on the pattern board corresponding to the image of the calibration image. After inputting all candidate bounding boxes and the first feature map into the center regression network, the center regression network can identify each candidate bounding box in the first feature map, thereby predicting the probability that each candidate bounding box contains the image center point of the pattern, as well as the coordinates of the image center point of the pattern, to further determine the coordinates of the image center point of each pattern in the calibration image. Specifically, the network structures of the center regression network and the bounding box regression network are as follows: Figure 3 As shown, both the center regression network and the bounding box regression network use five 3×3 convolution operations from the input layer to the output layer, and the ReLU function is used as the activation function.

[0065] The above technical solution involves designing a pattern board with multiple patterns on the calibration device's pattern plate. Calibration images are acquired for the calibration device, and then feature extraction is performed on these images to obtain a first feature map. This transforms the image data in the calibration image into a form that is easy to process and analyze. A deep learning architecture combining a bounding box regression network and a center regression network is constructed. This deep learning architecture extracts multiple regions of interest from the first feature map. Combined with pre-designed prior boxes, each candidate region of interest is output to obtain the coordinates of the image center point of each pattern in the calibration image. This method can obtain clear imaging edge information and, even when the calibration image has blurred edges or poor imaging quality, obtains more accurate candidate regions, significantly improving the detection accuracy of the image center point of the pattern.

[0066] In the embodiments of this application, inputting all candidate boxes into a central regression network to determine the center point of each pattern in the calibration image based on the predicted image coordinates output by the central regression network includes: inputting a first feature map and all candidate boxes into a central regression network to output multiple predicted image coordinates and a prediction probability corresponding to each preset image coordinate; arranging the multiple prediction probabilities from high to low, and determining the predicted image coordinates corresponding to the preset number of prediction probabilities at the top as the first image coordinates of the image center point of each pattern in the calibration image.

[0067] It can be understood that predicted image coordinates refer to the predicted coordinates of the image center point of the pattern on the pattern board for each candidate box in the first feature map, and the coordinate system is the image coordinate system of the calibration image. Prediction probability refers to the probability that the image center point of the pattern on the pattern board exists for each candidate box in the first feature map. Each candidate box outputs a predicted image coordinate and a predicted probability through a center regression network. The number of patterns on the pattern board of the calibration device is fixed. As can be seen from the aforementioned scheme, the number of candidate boxes for each region of interest is determined based on the number of patterns on the pattern board. Therefore, the number of predicted image coordinates determined based on the candidate boxes is greater than or equal to the number of patterns on the pattern board. Then, the obtained prediction probabilities can be arranged from high to low, and a preset number of prediction probabilities can be selected. Each predicted image coordinate corresponding to these prediction probabilities is determined as the first image coordinate of the image center point of each pattern in the calibration image.

[0068] In a specific embodiment, such as Figure 4 As shown, a ResNet50 residual network is used. The second feature map is output from each of ResNet50's stages 1, 2, 3, and 4. Each second feature map is processed by an upsampling module to obtain four third feature maps of the same size: F1, F2, F3, and F4. These four identical feature maps are concatenated along the channel dimension using feature concatenation to obtain a fourth feature map F5, thus fusing four feature maps of different scales to fully utilize information from different scales. Further, the fourth feature map is sequentially processed by a channel attention mechanism (CAM) and a spatial attention mechanism (SAM) to obtain a first feature map F6. This first feature map is then input into a PRPN (Preset Region Proposal Network). The PRPN network is also used to generate candidate regions. Specifically, as shown... Figure 5As shown, in the PRPN network, the first feature map F6 is input to the BoxRegression network. After being processed by NMS (Numerical Maximum Suppression), it outputs k regions of interest (ROIs) of the first feature map F6. The output channels of the BoxRegression network are k×4, including the (x, y, w, h) of k bounding boxes, where x, y, w, and h are the x-coordinate, y-coordinate, width, and height of the top-left corner of each ROI. Further, a preset prior box is generated based on the dimensions of the bounding rectangle of any pattern. Based on the preset prior box, the preset prior box and the ROIs are input to the RegionGenrate module to generate n candidate boxes for each ROI, where n is the total number of patterns on the pattern board. Further, the n candidate boxes generated above are input to the ROIPooling layer for pooling processing, and then input to the CenterRegression network to output the predicted image coordinates and predicted probability corresponding to each candidate box. The predicted image coordinates of the top n predicted probabilities are used as the first image coordinates of the image center point of the pattern in the calibration image. The output channel of the center regression network is n×k×3, containing (x, y, scores) points, where x, y, and scores are the horizontal and vertical coordinates of the predicted image and the predicted probability. Using this method, more accurate candidate regions can be obtained even in calibrated images with blurred edges and poor imaging quality, significantly improving the detection accuracy of pattern center points.

[0069] In embodiments of this application, the method further includes: sorting the actual center points of all patterns on the pattern board and obtaining the relative positions between all sorted actual center points; determining the world coordinates of each sorted actual center point based on the relative position of each sorted actual center point; performing perspective transformation on the first image coordinates of all image center points based on the relative positions of all sorted actual center points to determine the first image coordinates corresponding to each sorted actual center point; and determining the intrinsic parameters of the image acquisition device for acquiring calibration images based on the correspondence between the first image coordinates and world coordinates of each sorted actual center point, according to all the first image coordinates and world coordinates.

[0070] Camera intrinsic parameter calibration is a crucial step in computer vision, aiming to determine the camera's internal parameters, including focal length, optical center, and distortion parameters. These parameters are essential for subsequent image processing tasks such as stereo vision, 3D reconstruction, and object tracking. The actual center point refers to the center point of the pattern on the calibration device's pattern board in real space. Understandably, the image center point and the actual center point are relative; the image center point is the projection of the actual center point onto the calibration image. Using the aforementioned scheme, after obtaining the first image coordinates of the image center points of each pattern in the calibration image, the intrinsic parameters of the image acquisition device can be further calibrated by establishing the correspondence between the first image coordinates of the image center points and the world coordinates of the actual center points. However, the first image coordinates of these image center points obtained by the aforementioned scheme are an unordered set of data, making it impossible to determine the correspondence between the first image coordinates of each image center point and the world coordinates of each actual center point. Therefore, it is first necessary to sort these first image coordinates. Each sorted image center point is assigned a corresponding number, which distinguishes it from other image center points, allowing the retrieval of the world coordinates of its corresponding actual center point.

[0071] Specifically, the actual center points of all patterns on the pattern board can be sorted first. Since the positions of each pattern on the pattern board are fixed, the relative positions between the actual center points of all patterns on the pattern board can be obtained. Furthermore, after knowing the relative positions between the actual center points, the world coordinates of each actual center point can be directly derived. The coordinate system of this world coordinate system can use any fixed position in the space where the calibration device is located as the origin. Moreover, knowing the relative positions between the actual center points and their order, a perspective transformation can be performed on the first image coordinates of all image center points based on the relative positions of all sorted actual center points to find the correspondence between the actual center points and the image center points, thereby determining the first image coordinates corresponding to each sorted actual center point. Further, based on the correspondence between the first image coordinates and world coordinates of each sorted actual center point, the intrinsic parameters of the image acquisition device for acquiring and calibrating the image are determined according to all the first image coordinates and world coordinates. For example, the Zhang Zhengyou calibration method can be used to calculate the intrinsic parameters of the image acquisition device.

[0072] In the embodiments of this application, the arrangement of multiple patterns is an array, the relative positions include the array's arrangement interval, the number of rows in the array is a first quantity, and the number of columns in the array is a second quantity. Sorting the actual center points of all patterns includes: sorting the actual center points of all patterns based on the array in a row-first, column-later order; performing perspective transformation on the first image coordinates of all image center points based on the relative positions of all sorted actual center points to determine the first image coordinates corresponding to each sorted actual center point includes: after determining the first image coordinates of each image center point, determining two diagonal points among all image center points based on all first image coordinates; and determining the first image coordinates corresponding to the first image coordinates of each image center point based on the two diagonal points. Multiple first corner points of a target polygon; multiple second corner points corresponding to the shape of the pattern board are determined according to the array arrangement interval, a first quantity and a second quantity, so as to construct a second target polygon corresponding to the shape of the pattern board in the calibration image through the multiple second corner points; based on the array arrangement interval, second image coordinates corresponding to each sorted actual center point are generated in the second target polygon; a perspective transformation matrix between the first target polygon and the second target rectangle is determined according to the multiple first corner points and the multiple second corner points; each first image coordinate is transformed through the perspective transformation matrix to determine the second image coordinates corresponding to each first image coordinate, so as to determine the first image coordinates corresponding to each sorted actual center point.

[0073] In embodiments of this application, determining multiple first corner points for constructing a first target polygon in a calibration image based on two diagonal points includes: for any one of the two diagonal points, determining a line connecting each image center point to the diagonal point, and determining the number of patterns each line passes through; for each diagonal point, designating a first target line as the line passing through a first number of patterns, and designating a second target line as the line passing through a second number of patterns; for each diagonal point, determining the intersection point of the first target line and the second target line; and determining all diagonal points and all intersection points of the lines as multiple first corner points for constructing a first target polygon in the calibration image.

[0074] Specifically, to perform perspective transformation on the first image coordinates of all image center points, it is necessary to construct a viewing plane corresponding to all image center points, i.e., a first target polygon, and a viewing plane corresponding to all actual center points, i.e., a second target polygon. Furthermore, to facilitate the calibration of pattern center points, technicians can design the patterns on the calibration device's pattern plate as an array arrangement and obtain the number of rows and columns of the array, where the number of rows is the first quantity and the number of columns is the second quantity. After determining the first image coordinates of each image center point, the image center points are paired up, and the distance between the two image center points in the calibration image is calculated based on their first image coordinates. The pair of image center points with the largest distance is selected as the diagonal points. Taking any diagonal point, the line connecting each pattern's center point to that diagonal point is sequentially determined, and the number of times this line passes through the pattern is determined, i.e., the number of connected regions formed by the line passing through the pattern. The line that passes through the pattern a first number of times is designated as the first target line. For ease of understanding, we can assume the array is rectangular. If the line connecting a center point of an image to its opposite corner passes through a pattern equal to the number of rows in the array, then this center point is highly likely to be a corner of the rectangle, and the first target line is highly likely to be a side of the rectangle. Similarly, the second target line passing through a pattern equal to the second number is also highly likely to be another side of the rectangle. Further, we determine the intersection points of all first target lines and all second target lines—that is, the intersection points of the first target line connecting one opposite corner with the first target line connecting the other opposite corner. Multiple lines can have multiple intersection points. These intersection points are the corner points of the array, including the opposite corners mentioned earlier. The first corner point refers to the corner point used to construct the first target polygon. By determining all the opposite corners and their intersection points as the first corner points, and sorting these first corner points in clockwise order, connecting them sequentially, the first target polygon can be constructed.

[0075] Furthermore, by using the aforementioned method, the diagonal point with the largest distance is found based on the arrangement interval of the pattern plate array. Based on this maximum distance value, the number of rows and columns of the array, multiple second corner points corresponding to the shape of the pattern plate can be determined. These second corner points are then used to construct a second target polygon corresponding to the shape of the pattern plate in the calibration image. That is, a view plane corresponding to all actual center points is constructed in the calibration image. Furthermore, within the second target polygon, given the arrangement interval of the array and the order of the actual center points, second image coordinates corresponding to each sorted actual center point can be generated in the calibration image. Then, based on the multiple first corner points and multiple second corner points, the perspective transformation matrix between the first target polygon and the second target rectangle can be determined. That is, the perspective transformation matrix between the two view planes. After transforming each first image coordinate through the perspective transformation matrix, second image coordinates identical to the perspective-transformed coordinates of the first image coordinates can be found. That is, the second image coordinates corresponding to each first image coordinate are found. Therefore, given the order of the actual center points corresponding to the second image coordinates, the first image coordinates corresponding to each sorted actual center point can be determined. Therefore, the coordinates of the center points of the patterns on the two view planes can be mapped one-to-one, and the correspondence between the first image coordinates and the world coordinates can also be determined. Based on the above correspondence, and according to all the first image coordinates and world coordinates, the intrinsic parameters of the image acquisition device for acquiring and calibrating the images can be determined.

[0076] Figure 6 A schematic diagram illustrating the structure of an artificial intelligence-based image processing system according to an embodiment of this application is shown. Figure 6 As shown, an artificial intelligence-based image processing system is provided, including:

[0077] Image acquisition device 610 is used to acquire calibration images of the calibration device;

[0078] The calibration device 620 includes a pattern plate with multiple patterns;

[0079] The image processing apparatus 630 is configured to perform the above-described artificial intelligence-based image processing method.

[0080] Specifically, the image acquisition device can be an optical camera, an infrared camera, a scanner, etc. The calibration image can be an optical image or an infrared image. The calibration device includes a pattern board with multiple patterns. The image acquisition device can acquire calibration images of the calibration device. The patterns on the pattern board differ from other areas in shape, color, reflectivity, or temperature. For example, when the image acquisition device is an infrared camera, the pattern in the acquired infrared image shows a brightness difference compared to the background. When the image acquisition device is an optical camera, the pattern in the acquired optical image shows a significant color contrast with the background. The image processing device can process the acquired calibration image using the above image processing methods to obtain the coordinates of the image center point of the pattern.

[0081] In the embodiments of this application, the calibration device further includes a heating plate connected to the pattern plate, such that when the heating plate is heated, each pattern in the calibration image presents a clear contrast with the background. The image acquisition device includes an infrared camera and a visible light camera, and the multiple patterns are multiple identical hollow patterns arranged in an array.

[0082] like Figure 7 As shown, the calibration device includes a heating plate 10 and a pattern plate 20, with multiple perforated patterns 30 designed on the pattern plate 20. The heating plate has high structural strength and is not easily bent or deformed. It also has a temperature control system to stabilize the temperature at a set value. The heating plate is rectangular in shape with mounting holes at its four corners for connecting to the pattern plate. The pattern plate has the same dimensions and mounting holes as the heating plate, and the perforated patterns are regularly arranged circles, with at least four circles. The heating plate and pattern plate are fixed together with bolts, leaving a gap between them rather than being tightly attached. Once the two plates are fixed, the device is ready to use when powered on. When powered on, the heating plate heats up, and in the calibration image captured by the infrared thermal imaging camera, the perforated patterns on the pattern plate appear bright. Furthermore, the heating plate and pattern plate are two contrasting colors, resulting in a clear contrast between the perforated patterns and the background in the calibration image captured by the visible light camera.

[0083] Figure 8 The illustration shows a schematic diagram of the process for determining camera intrinsic parameters according to an embodiment of this application. Figure 8 As shown, this method for determining camera intrinsic parameters is applied to, for example... Figure 6 The image acquisition system shown uses... Figure 7The calibration device is shown. First, power on the calibration device and camera, setting the temperature to T1℃, which is higher than the ambient temperature. Place the calibration device on the ground. Mount the camera vertically downwards. After powering on the camera, acquire calibration images. Adjust the camera's position so that the calibration device occupies approximately one-third of the camera's field of view. Move the calibration device while simultaneously acquiring calibration images, adjusting its position so that it appears in every area of ​​the camera's field of view. Acquire approximately 20 images to form an image set. For each calibration image, use the deep learning algorithm described above in the image processing method to calculate the first image coordinates of the circular image center point in the calibration image. Further sort the first image coordinates using the aforementioned method to obtain the order of the circular center points, thus mapping the first image coordinates to the world coordinates of the actual center point, allowing the calculation of the camera's intrinsic parameters using the Zhang Zhengyou calibration method.

[0084] Step 1: Calculate the distance between each pair of center points in the image, and take the maximum distance. max A set of points is taken as diagonal points p1 and p2.

[0085] Step 2: Take a point p_diagonal from the diagonal points, and successively calculate the straight line L formed by p_diagonal and the center points of the image. i , and L i The number N of connected regions traversed by the hollowed-out pattern li .

[0086] Step 3: Given that the number of regularly arranged circles is r rows and c columns, take N... li The straight line L = r r Take N li The straight line L = c c .

[0087] Step 4: Perform steps 2) and 3) on the two diagonal points to obtain two sets of straight lines L. r1 L c1 and L r2 L c2 Find L r1 and L c2 The intersection point p3, L r2 and L c1 The intersection point p4.

[0088] Step 5: Reorder the four points p1, p2, p3, and p4 in clockwise order to obtain the polygon containing the center point of the image. src The four corner points src ;

[0089] Step 6: Construct a regular rectangle (polygon) real Among them, the coordinates of the four corner points are...real (0, 0), Based on the arrangement of the hollow pattern, the row-first, column-second calculation is performed on the polygon. real The second image coordinates of each sorted actual center point are center real ;

[0090] Step 7: According to corner src and corner real Find the quadrilateral polygon src To rectangle polygon real perspective transformation matrix M p ;

[0091] Step 8: Sequentially transform the first image coordinates of the image center point through the perspective transformation matrix M. p The transformation yields new coordinates, corresponding to the second image coordinates (center) of each sorted actual center point. real To determine the first image coordinates corresponding to each sorted actual center point;

[0092] Step 9: Based on the arrangement interval of the hollow pattern, generate the world coordinates (center) of the actual center point of each sorted element in the order of row first, then column. world ;

[0093] Step 10: Based on the world coordinates (center) of each sorted actual center point world Using the coordinates of the first image, the camera's intrinsic parameters are calculated using the Zhang Zhengyou calibration method.

[0094] Figure 9 A schematic diagram illustrating a structural block diagram of an artificial intelligence-based image processing apparatus according to an embodiment of this application is shown. Figure 9 As shown, this application provides an image processing device based on artificial intelligence, which may include:

[0095] Memory 910 is configured to store instructions; and

[0096] The processor 920 is configured to retrieve instructions from the memory 910 and to implement the aforementioned image processing method when executing the instructions.

[0097] Specifically, in this embodiment of the application, the processor 920 can be configured to:

[0098] Acquire calibration images from the calibration device, which includes a pattern plate with multiple patterns;

[0099] Feature extraction is performed on the calibration image to obtain the processed first feature map;

[0100] The first feature map is input into the bounding box regression network, so that the bounding box regression network outputs multiple regions of interest in the first feature map that correspond to the calibration device.

[0101] Based on the preset prior bounding boxes, a preset number of candidate bounding boxes are generated for each region of interest. The size of each candidate bounding box is the same as the size of the preset prior bounding box, and the preset number is determined according to the number of multiple patterns.

[0102] All candidate boxes and the first feature map are input into the center regression network to obtain the coordinates of the center point of each pattern in the calibration image.

[0103] In this embodiment of the application, the processor 920 can also be configured to:

[0104] The first feature map and all candidate boxes are input into the center regression network to output multiple predicted image coordinates and the prediction probability corresponding to each preset image coordinate. The multiple prediction probabilities are arranged from high to low, and the predicted image coordinates corresponding to the preset number of prediction probabilities are determined as the first image coordinates of the image center point of each pattern in the calibration image.

[0105] In the embodiments of this application, the multiple patterns are multiple identical and regularly arranged patterns, and the processor 820 can also be configured to:

[0106] Obtain the size of the bounding rectangle of any pattern; generate a preset prior box based on the size of the bounding rectangle; for each region of interest, input the preset prior box and the region of interest into the region generation module, so that the region generation module can output a preset number of candidate boxes for each region of interest.

[0107] In embodiments of this application, the processor 920 may also be configured to:

[0108] The function expression for the region generation module is shown in the following formula (1):

[0109]

[0110] Among them, Proposal (k，n) It refers to the nth candidate box in the kth region of interest, bbox. k It refers to the k-th region of interest, Base_anchor n It refers to the nth candidate box generated based on the preset prior box. x1 and x2 are the x-coordinates of the diagonal points of the bounding rectangle corresponding to the preset prior box, respectively. y1 and y2 are the y-coordinates of the diagonal points of the bounding rectangle corresponding to the preset prior box, respectively. x and y are the x-coordinates and y-coordinates of the fixed points of each region of interest, respectively. w and h are the width and height of each region of interest, respectively.

[0111] In embodiments of this application, the processor 920 may also be configured to:

[0112] The calibration image is input into the residual network, and the residual network outputs multiple second feature maps of different scales. The multiple second feature maps are input into the upsampling module, and the upsampling module outputs multiple third feature maps of the same size. All the third feature maps are concatenated to obtain a fourth feature map. The fourth feature map is input into the attention mechanism module, and the attention mechanism module outputs a first feature map after pooling the fourth feature map in both spatial and channel dimensions.

[0113] In embodiments of this application, the processor 920 may also be configured to:

[0114] The actual center points of all patterns on the pattern board are sorted, and the relative positions between all sorted actual center points are obtained. The world coordinates of each sorted actual center point are determined according to the relative position of each sorted actual center point. Based on the relative positions of all sorted actual center points, the first image coordinates of all image center points are transformed by perspective to determine the first image coordinates corresponding to each sorted actual center point. Based on the correspondence between the first image coordinates and world coordinates of each sorted actual center point, the intrinsic parameters of the image acquisition device for acquiring calibration images are determined according to all the first image coordinates and world coordinates.

[0115] In embodiments of this application, the arrangement of multiple patterns is an array, the relative positions include the array spacing, the number of rows in the array is a first quantity, the number of columns in the array is a second quantity, and the processor 920 can also be configured to:

[0116] The actual center points of all patterns are sorted according to the array in row-first, column-second order; and

[0117] After determining the first image coordinates of each image center point, two diagonal points among all image center points are determined based on all first image coordinates. Multiple first corner points are determined based on these two diagonal points to construct a first target polygon in the calibration image. Multiple second corner points corresponding to the shape of the pattern board are determined based on the array's arrangement interval, a first quantity, and a second quantity, so as to construct a second target polygon corresponding to the shape of the pattern board in the calibration image using these multiple second corner points. Based on the array's arrangement interval, second image coordinates corresponding to each sorted actual center point are generated in the second target polygon. A perspective transformation matrix between the first target polygon and the second target rectangle is determined based on the multiple first corner points and multiple second corner points. Each first image coordinate is transformed using the perspective transformation matrix to determine the second image coordinates corresponding to each first image coordinate, thereby determining the first image coordinates corresponding to each sorted actual center point.

[0118] In embodiments of this application, the processor 920 may also be configured to:

[0119] For any one of the two diagonal points, determine the line connecting each center point and the diagonal point to determine the number of patterns each line passes through; for each diagonal point, designate the line that passes through a first number of patterns as the first target line, and designate the line that passes through a second number of patterns as the second target line; determine the intersection points of all the first target lines and all the second target lines; and determine all the diagonal points and all the intersection points as multiple first corner points for constructing the first target polygon in the calibration image.

[0120] The above technical solution involves designing the calibration device's pattern plate with multiple hollowed-out patterns, and superimposing heating plates of different colors onto the pattern plate. When the heating plates are powered on, the colors and temperatures of the patterns on the pattern plate differ from other areas in the calibration image acquired by the image acquisition device, making the patterns clearly distinguishable from the background. Then, a residual network is used to extract features from the calibration image to obtain a first feature map, converting the image data in the calibration image into a form that is easy to process and analyze. A deep learning architecture combining a bounding box regression network and a center regression network is constructed. This deep learning architecture extracts multiple regions of interest from the first feature map, and further combines this with a preset prior box designed based on the pattern size to output each candidate region of interest, along with its corresponding predicted image coordinates and predicted probability. The coordinates of the image center point of each pattern in the calibration image are obtained based on the predicted probability and the number of patterns. This method can obtain more accurate candidate regions even when the calibration image has blurred edges and poor image quality, significantly improving the detection accuracy of the image center point of the pattern. Furthermore, a viewing plane is constructed based on the actual relative position of the pattern plate, and another viewing plane is constructed based on the coordinates of the image center point of the calibration image, resulting in a perspective transformation matrix between the two viewing planes. In this way, the first image coordinates of each center point can be correlated with world coordinates, thereby obtaining the camera's intrinsic parameters through Zhang Zhengyou's calibration method. The above method, through the design and structure of the calibration device, enables the imaging of the calibration image, and employs a unified algorithm compatible with both visible light cameras and infrared thermal imaging cameras. The combination of these two methods not only yields clear imaging edge information but also provides more accurate candidate regions even in cases of blurred edges and poor imaging quality in the calibration image, significantly improving the detection accuracy of image recognition.

[0121] This application also provides a machine-readable storage medium storing instructions that cause a machine to execute the above-described artificial intelligence-based image processing method.

[0122] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 10 As shown, the computer device includes a processor A01, a network interface A02, memory (not shown), and a database (not shown) connected via a system bus. The processor A01 provides computing and control capabilities. The memory includes internal memory A03 and a non-volatile storage medium A04. The non-volatile storage medium A04 stores an operating system B01, a computer program B02, and a database (not shown). The internal memory A03 provides an environment for the operation of the operating system B01 and the computer program B02 stored in the non-volatile storage medium A04. The database stores image processing data based on artificial intelligence. The network interface A02 communicates with external terminals via a network connection. When the computer program B02 is executed by the processor A01, it implements an image processing method based on artificial intelligence.

[0123] Those skilled in the art will understand that Figure 10 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0124] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0125] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0126] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0127] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0128] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0129] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0130] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0131] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0132] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. An image processing method, characterized in that, The method includes: Acquire calibration images from a calibration device, the calibration device comprising a pattern plate with multiple patterns; Feature extraction is performed on the calibration image to obtain the processed first feature map; The first feature map is input into a bounding box regression network to output multiple regions of interest in the first feature map corresponding to the calibration device. Based on a preset prior bounding box, a preset number of candidate bounding boxes are generated for each region of interest. The size of each candidate bounding box is the same as the size of the preset prior bounding box. The preset number is determined according to the number of the multiple patterns. All candidate boxes and the first feature map are input into the center regression network to obtain the coordinates of the image center point of each pattern in the calibration image; The actual center points of all patterns are sorted according to the row-first-column order based on the array, and the relative positions between the sorted actual center points are obtained. The arrangement of the multiple patterns is an array, and the relative positions include the arrangement interval of the array. The number of rows of the array is a first quantity, and the number of columns of the array is a second quantity. Determine the world coordinates of each sorted actual center point based on its relative position. Based on the relative positions of all sorted actual center points, perform perspective transformation on the first image coordinates of all image center points to determine the first image coordinates corresponding to each sorted actual center point. Based on the correspondence between the first image coordinates and world coordinates of each sorted actual center point, the intrinsic parameters of the image acquisition device for acquiring the calibration image are determined according to all the first image coordinates and world coordinates. The step of performing perspective transformation on the first image coordinates of all image center points based on the relative positions of all sorted actual center points to determine the first image coordinates corresponding to each sorted actual center point includes: After determining the first image coordinates of each image center point, determine two diagonal points among all image center points based on all the first image coordinates; Based on the two diagonal points, a plurality of first corner points are determined in the calibration image to construct the first target polygon; Based on the array arrangement interval, the first quantity, and the second quantity, a plurality of second corner points corresponding to the shape of the pattern plate are determined, so as to construct a second target polygon corresponding to the shape of the pattern plate in the calibration image through the plurality of second corner points; Based on the arrangement interval of the array, second image coordinates corresponding to each sorted actual center point are generated in the second target polygon; Determine the perspective transformation matrix between the first target polygon and the second target polygon based on the plurality of first corner points and the plurality of second corner points; Each first image coordinate is transformed by the perspective transformation matrix to determine the second image coordinate corresponding to each first image coordinate, thereby determining the first image coordinate corresponding to each sorted actual center point.

2. The image processing method according to claim 1, characterized in that, The step of inputting all candidate boxes into the center regression network to determine the image center point of each pattern in the calibration image based on the predicted image coordinates output by the center regression network includes: The first feature map and all candidate boxes are input into the central regression network to output multiple predicted image coordinates and the prediction probability corresponding to each preset image coordinate. The predicted probabilities are arranged from high to low, and the predicted image coordinates corresponding to the first preset number of predicted probabilities are determined as the first image coordinates of the image center point of each pattern in the calibration image.

3. The image processing method according to claim 1, characterized in that, The multiple patterns are multiple identical patterns arranged in a regular pattern. The step of generating a preset number of candidate boxes for each region of interest based on preset prior boxes includes: Get the size of the bounding rectangle of any pattern; The preset prior box is generated based on the dimensions of the circumscribed rectangle; For each region of interest, the preset prior bounding box and the region of interest are input to the region generation module, so that the region generation module outputs a preset number of candidate bounding boxes for each region of interest.

4. The image processing method according to claim 3, characterized in that, The function expression of the region generation module is shown in the following formula (1): （1） in, It refers to the nth candidate box in the kth region of interest. It refers to the k-th region of interest. This refers to the nth candidate box generated based on the preset prior box. , and refer to the x-coordinates of the diagonal points of the bounding rectangle corresponding to the preset prior box, respectively. These refer to the ordinates of the diagonal points of the bounding rectangle corresponding to the preset prior box. y refers to the x-coordinate and y-coordinate of a fixed point in each region of interest, while w and h refer to the width and height of each region of interest, respectively.

5. The image processing method according to claim 1, characterized in that, The process of processing the calibration image to obtain the processed first feature map includes: The calibration image is input into the residual network, and the residual network outputs multiple second feature maps of different scales. Multiple second feature maps are input into the upsampling module, and multiple third feature maps of the same size are output through the upsampling module; Connect all the third feature maps to obtain the fourth feature map; The fourth feature map is input into the attention mechanism module, and the attention mechanism module outputs the first feature map after pooling the fourth feature map in terms of spatial and channel dimensions.

6. The image processing method according to claim 1, characterized in that, The step of determining multiple first corner points for constructing the first target polygon in the calibration image based on the two diagonal points includes: For any one of the two diagonal points, determine the line connecting each center point to the diagonal point, so as to determine the number of times each line passes through the pattern; For each diagonal point, the line that passes through the pattern a first number of times is designated as the first target line, and the line that passes through the pattern a second number of times is designated as the second target line; Determine the intersection points of all lines connecting the first target and all lines connecting the second target; All diagonal points and all intersections of the connecting lines are determined as multiple first corner points for constructing the first target polygon in the calibration image.

7. An image processing apparatus, characterized in that, include: The memory is configured to store instructions; as well as The processor is configured to retrieve the instructions from the memory and, when executing the instructions, to implement the image processing method according to any one of claims 1 to 6.

8. An image processing system, characterized in that, include: Image acquisition equipment, used to acquire calibration images from the calibration device; The calibration device includes a pattern plate with multiple patterns; The image processing apparatus according to claim 7.

9. The image processing system according to claim 8, characterized in that, The calibration device also includes a heating plate connected to the pattern plate, so that when the heating plate is heated, each pattern in the calibration image presents a clear contrast with the background. The image acquisition device includes an infrared camera and a visible light camera. The multiple patterns are multiple identical hollow patterns arranged in an array.

10. A machine-readable storage medium, characterized in that, The machine-readable storage medium stores instructions for causing the machine to perform the image processing method according to any one of claims 1 to 6.

Images