Sheath character recognition method and system based on domestic environment

Abstract

The invention discloses a sheath character recognition method and system based on a domestic environment, and relates to the technical field of cable quality detection, and the method comprises the steps: obtaining an image containing cable code spraying characters in real time, and carrying out the segmentation preprocessing; tCP connection is established between the client and the server, and the client transmits the preprocessed image to the server; the server sends the image into the segmentation model for reasoning by calling the SDK of the domestic board card; the server side returns a reasoning result to the client side, then the client side extracts a point set of the most appropriate contour where the character is located, the area where the character is located is cut out from the original image according to the point set, correction and recognition preprocessing are conducted on the area, and then the area is sent into a segmentation model for reasoning; and the server transmits the reasoning result to the client, and the client compares the reasoning result with a preset dictionary table to obtain an identification result. According to the invention, the problems of low efficiency, high cost and the like caused by incapability of correcting code spraying errors of the code spraying machine in time are solved.

Description

Technical Field

[0001] The invention relates to the technical field of cable quality detection, in particular to a sheath character recognition method and system based on a domestic environment. Background Art

[0002] Sheathed cable character recognition is a production method that has emerged with the development of industrial technology. Its purpose is to identify cable characters through automated equipment, and to determine whether the cable is painted normally by comparing the identified characters with the preset characters, so as to adjust the inkjet printer in time, reduce the generation rate of waste wires, and reduce corporate losses.

[0003] As the second largest industry in my country, the cable industry needs to use machine vision, automation technology, and Internet of Things technology to accelerate the upgrading of the cable manufacturing industry, improve production efficiency and manufacturing level, and reduce manufacturing costs. This is an urgent problem to be solved. As an important component of cable quality control, fully automatic cable testing equipment has received more and more attention in recent years. As an important part of cable testing projects, cable appearance testing has always been a hot topic of research at home and abroad.

[0004] In the cable manufacturing process, the cable production date, production batch, cable type, length and other information are usually marked on the cable surface by inkjet printing, laser printing and other methods. In order to control product quality and record product information, it is necessary to identify and monitor the cable surface information. Traditional cable surface information identification often relies on workers' naked eye observation, which is labor-intensive and inefficient. At the same time, with the continuous increase in cable production speed, the current cable manufacturing speed is generally 60m / min, and it is impossible to achieve real-time monitoring of cable surface information by manual methods. Therefore, real-time recognition of cable surface characters in high-speed motion is of great significance for improving cable production efficiency, reducing costs, and achieving industrial upgrading. Summary of the invention

[0005] The present invention aims to solve the problems in the sheath cable industry where the human eye cannot observe the inkjet characters in real time, resulting in low efficiency and high cost due to the inability to promptly correct inkjet printer coding errors. The present invention provides a sheath character recognition method and system based on a domestic environment.

[0006] In the first aspect, the present invention provides a sheath character recognition method based on a domestic environment, and the technical solution adopted to solve the above technical problems is as follows:

[0007] A sheath character recognition method based on a domestic environment comprises the following steps:

[0008] S1, using a 2D camera to obtain images containing cable coding characters in real time;

[0009] S2, calling the camera function through the client interface to obtain the image and perform segmentation preprocessing;

[0010] S3, a TCP connection is established between the client and the server, and then the image data after segmentation preprocessing is transmitted to the server; the server receives the image data and stores it in a queue, and the server calls the SDK of the domestic board to take the image data out of the queue and send it to the segmentation model for reasoning;

[0011] S4, after the server completes the reasoning, it returns the result to the client, and the client then performs post-segmentation processing to extract the point set of the most appropriate contour of the character, and cuts out the area where the character is located from the image in step S1 according to the point set;

[0012] S5, correct and identify the cropped area image, then repeat step S3, and the server calls the SDK for inference;

[0013] S6. The server transmits the inference result to the client through TCP. The client compares the inference result with the preset dictionary table to obtain the recognition result.

[0014] Optionally, execute step S2 to acquire an image and perform segmentation preprocessing, specifically including adjusting the image size, changing the data type of the pixel points, and normalizing the numerical values, so as to obtain standardized image data.

[0015] Optionally, the segmentation model involved is a computing model based on a deep learning or machine learning algorithm, which is used to process and analyze image data to identify and segment the coded characters in the image;

[0016] The training process of the segmentation model involved is as follows:

[0017] Collect images containing cable coding characters as training data;

[0018] Perform image enhancement, normalization, and data enhancement preprocessing operations on the collected training data;

[0019] According to the application scenario and requirements of the image segmentation task, select one of U-Net, FCN, SegNet, DeepLab and OCR as the basic architecture of the segmentation model;

[0020] The segmentation model of the selected infrastructure is trained using the training data after the preprocessing operation. During the training process, the model parameters are optimized by the back propagation algorithm to minimize the loss function so that the segmentation model can accurately segment the characters in the image.

[0021] Evaluate the performance of the trained segmentation model on an independent validation set and fine-tune the model based on the evaluation results;

[0022] Deploy the tuned segmentation model to actual application scenarios to achieve real-time character recognition and segmentation functions.

[0023] Optionally, the step S4 involved specifically includes:

[0024] S4.1, Filtering: The image data sent to the segmentation model includes the foreground and background. The segmentation model calculates each pixel in the image data and estimates the probability that the pixel belongs to the foreground. After all calculations are completed, the segmentation model returns a probability matrix of the same size as the image data. Then, a threshold t is set to convert the probability matrix into a binary form to obtain a binary image C after preliminary segmentation. The specific formula is as follows:

[0025] C=t*O,

[0026] Wherein, C represents the binary image after preliminary segmentation, t represents the set threshold, and O represents the image data obtained in step S1;

[0027] S4.2, dilation: select a 2x2 dilation kernel, dilate the binary image after preliminary segmentation, and connect the independent pixels in the foreground;

[0028] S4.3, determine the bounding rectangle: first use the connected component labeling algorithm CCL to identify the connected pixel area in the image, mark each connected pixel area as an independent foreground object, and calculate its minimum rectangular bounding box, then find the minimum bounding rectangle in any direction through the rotating caliper algorithm, and select the rectangle with the smallest area as the final bounding rectangle, finally determine the final bounding rectangle as the best rectangular bounding rectangle through principal component analysis, and apply it to the image data obtained in step S1;

[0029] S4.4, sorting: for each determined bounding rectangle, determine the x-coordinate of its leftmost pixel, create an array to store each bounding rectangle and its corresponding x-coordinate of the leftmost pixel, use the sorting function provided by the programming language to sort the entire array in ascending order according to the x-coordinate of the leftmost pixel, and process each bounding rectangle according to the sorting result to ensure that the bounding rectangle corresponds to its foreground object one by one;

[0030] S4.5, prevention of cross-frames: for each determined bounding rectangle, compare the y coordinates of its adjacent rectangle on the right. If the y coordinate of the rightmost pixel of one bounding rectangle is greater than the y coordinate of the leftmost pixel of its right bounding rectangle, it means that the two bounding rectangles overlap in the vertical direction. Then, move the right side of the left bounding rectangle to the right, or adjust the left side of the right bounding rectangle to the left, to ensure that the two optimal bounding rectangles do not overlap in the vertical direction. After adjustment, check each bounding rectangle again to confirm that no overlap occurs.

[0031] S4.6, image cropping: obtain the coordinates of the four vertices of each circumscribed rectangle to define the position of each character in the image data obtained in step S1, and crop the area in the image where the character is located.

[0032] Optionally, executing step S5, performing correction and recognition preprocessing on the cropped area image, specifically including:

[0033] S5.1, correcting the cropped area image by perspective transformation technology;

[0034] S5.2. Scale the corrected image to the size of the segmentation model input: first, keep the height of the corrected image unchanged, and scale the width to 2 / 3 of the original width; then calculate the ratio R, which is the ratio of the width to the height in the corrected image, and the formula is as follows: R = (img.cols) / (img.rows); then, for the image after the initial scaling, determine the product of its height imgH and R and its width imgW. If the product of its height imgH and R is greater than its width imgW, the scaled width value resize_W of the image is the width imgW; if the product of its height imgH and R does not exceed its width imgW, the scaled width value resize_W of the image is the product of its height imgH and R. According to the judgment result, perform secondary scaling of the image; after completing the secondary scaling, if the image width still does not meet the input standard of the segmentation model, fill it with grayscale value;

[0035] S5.3. Convert the data type of the scaled image to a 32-bit floating point number, calculate the average of all pixel values ​​of the converted image, then divide each pixel value of the converted image by 255 and subtract the average of all pixel values, and finally divide by the variance to complete the normalization of the scaled image.

[0036] In the second aspect, the present invention provides a sheath character recognition system based on a domestic environment, and the technical solution adopted to solve the above technical problems is as follows:

[0037] A sheath character recognition system based on a domestic environment, the structure of which includes:

[0038] An image acquisition module is used to acquire images containing cable coding characters in real time through a 2D camera;

[0039] The segmentation preprocessing module is used to call the camera function through the client interface to obtain images and perform segmentation preprocessing;

[0040] A connection establishment module is used to establish a TCP connection between the client and the server to transmit the segmentation pre-processed image data to the server;

[0041] The data transmission module is used to receive image data from the server and store it in a queue. By calling the SDK of the domestic board, the image data is taken out of the queue and sent to the segmentation model for reasoning;

[0042] The result return module is used to return the result to the client after the segmentation model completes the inference;

[0043] The extraction and cutting module is used to perform post-segmentation processing on the client side, extract the point set of the most appropriate contour where the character is located, and cut out the area where the character is located from the image obtained by the image acquisition module according to the point set;

[0044] The correction processing module is used to correct and identify the cropped area image, and then call the connection establishment module and the data transmission module to reason about the area image through the segmentation model;

[0045] The comparison and recognition module is used to obtain the inference result of the image of the area returned to the client by the result return module, and compare the inference result with the preset dictionary table to obtain the recognition result and transmit it to the client for display.

[0046] Optionally, the segmentation preprocessing module involved calls the camera function through the client interface to obtain the image and perform segmentation preprocessing, specifically including adjusting the image size, changing the data type of the pixel points and normalizing the numerical size, so as to obtain standardized image data.

[0047] Optionally, the segmentation model involved is a computing model based on a deep learning or machine learning algorithm, which is used to process and analyze image data to identify and segment the coded characters in the image;

[0048] The training process of the segmentation model involved is as follows:

[0049] Collect images containing cable coding characters as training data;

[0050] Perform image enhancement, normalization, and data enhancement preprocessing operations on the collected training data;

[0051] According to the application scenario and requirements of the image segmentation task, select one of U-Net, FCN, SegNet, DeepLab and OCR as the basic architecture of the segmentation model;

[0052] The segmentation model of the selected infrastructure is trained using the training data after the preprocessing operation. During the training process, the model parameters are optimized by the back propagation algorithm to minimize the loss function so that the segmentation model can accurately segment the characters in the image.

[0053] Evaluate the performance of the trained segmentation model on an independent validation set and fine-tune the model based on the evaluation results;

[0054] Deploy the tuned segmentation model to actual application scenarios to achieve real-time character recognition and segmentation functions.

[0055] Optionally, the extraction and cropping module involved performs post-segmentation processing on the client to extract a point set of the most appropriate contour of the character, and crop the area where the character is located from the image obtained by the image acquisition module according to the point set. The specific process includes:

[0056] Filtering: The image data sent to the segmentation model includes the foreground and background. The segmentation model calculates each pixel in the image data and estimates the probability that the pixel belongs to the foreground. After all calculations are completed, the segmentation model returns a probability matrix of the same size as the image data. Then, a threshold t is set to convert the probability matrix into a binary form to obtain a binary image C after preliminary segmentation. The specific formula is as follows:

[0057] C=t*O,

[0058] Wherein, C represents the binary image after preliminary segmentation, t represents the set threshold, and O represents the image data obtained in step S1;

[0059] Dilation: Select a 2x2 dilation kernel to dilate the binary image after preliminary segmentation to connect the independent pixels in the foreground;

[0060] Determine the bounding rectangle: first use the connected component labeling algorithm CCL to identify the connected pixel area in the image, mark each connected pixel area as an independent foreground object, and calculate its minimum rectangular bounding box, then find the minimum bounding rectangle in any direction through the rotating caliper algorithm, and select the rectangle with the smallest area as the final bounding rectangle, and finally determine the final bounding rectangle as the best rectangular bounding rectangle through principal component analysis, and apply it to the image data obtained in step S1;

[0061] Sorting: For each bounding rectangle, determine the x-coordinate of its leftmost pixel, create an array to store each bounding rectangle and its corresponding x-coordinate of the leftmost pixel, use the sorting function provided by the programming language to sort the entire array in ascending order according to the x-coordinate of the leftmost pixel, and process each bounding rectangle according to the sorting result to ensure that the bounding rectangle corresponds to its foreground object one by one;

[0062] Prevent cross-frames: For each bounding rectangle determined, compare the y coordinates of its adjacent rectangle on the right. If the y coordinate of the rightmost pixel of one bounding rectangle is greater than the y coordinate of the leftmost pixel of its right bounding rectangle, it means that the two bounding rectangles overlap in the vertical direction. Then move the right side of the left bounding rectangle to the right, or adjust the left side of the right bounding rectangle to the left to ensure that the two optimal bounding rectangles do not overlap in the vertical direction. After adjustment, check each bounding rectangle again to confirm that there is no overlap.

[0063] Image cropping: Obtain the coordinates of the four vertices of each circumscribed rectangle to define the position of each character in the image data obtained in step S1, and crop the area in the image where the character is located.

[0064] Optionally, the correction processing module performs correction and recognition preprocessing on the cropped area image, and the specific process includes:

[0065] Correct the cropped area image through perspective transformation technology;

[0066] Scale the corrected image to the size of the segmentation model input: first, keep the height of the corrected image unchanged, and scale the width to 2 / 3 of the original width; then calculate the ratio R, which is the ratio of the width to the height in the corrected image, and the formula is as follows: R = (img.cols) / (img.rows); then, for the image after the initial scaling, determine the product of its height imgH and R and its width imgW. If the product of its height imgH and R is greater than its width imgW, the scaled width value resize_W of the image is the width imgW; if the product of its height imgH and R does not exceed its width imgW, the scaled width value resize_W of the image is the product of its height imgH and R. According to the judgment result, perform secondary scaling of the image; after completing the secondary scaling, if the image width still does not meet the input standard of the segmentation model, fill it with grayscale value;

[0067] Convert the data type of the scaled image to a 32-bit floating point number, calculate the average of all pixel values ​​of the converted image, then divide each pixel value of the converted image by 255 and subtract the average of all pixel values, and finally divide by the variance to complete the normalization of the scaled image.

[0068] The present invention provides a method and system for identifying jacket characters based on a domestic environment, which has the following beneficial effects compared with the prior art:

[0069] 1. The present invention can complete the character recognition function of the sheathed cable by using a 2D camera in conjunction with a corresponding segmentation model. By comparing the recognized characters with the dictionary table, it can be found whether the paint characters of the sheathed cable are accurate, which can improve product quality and reduce losses caused by spray gun errors;

[0070] 2. The client and server of the present invention are connected via TCP, and can respond quickly to meet the requirements of fast processing of images or videos; by analyzing the post-segmentation processing data, filtering, dilation, finding the best circumscribed rectangle, sorting, avoiding cross frames and other operations are performed in sequence, which can better realize the segmentation of spray-painted characters while ensuring a certain speed; through a series of operations of image correction and scaling of the cropped image, the spray-painted characters appear in the appropriate position in the cropped characters and the spacing between the spray-painted characters in the scaled image is reasonable, which can improve the recognition accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] Attached Figure 1 is a flow chart of a method according to Embodiment 1 of the present invention;

[0072] Attached Figure 2 This is a module connection block diagram of the second embodiment of the present invention. DETAILED DESCRIPTION

[0073] In order to make the technical solution, the technical problem solved and the technical effect of the present invention more clearly understood, the technical solution of the present invention is clearly and completely described below in conjunction with specific embodiments.

[0074] Embodiment 1:

[0075] Combined with Figure 1 This embodiment proposes a sheath character recognition method based on a domestic environment, comprising the following steps:

[0076] S1. Use a 2D camera to obtain images containing cable coding characters in real time.

[0077] When implementing this step, a 2D industrial camera needs to be fixed above the cable to be tested to ensure that the camera's field of view includes the coded characters of the cable to be tested.

[0078] S2. Call the camera function through the client interface to obtain the image and perform segmentation preprocessing, including adjusting the image size, changing the data type of the pixel points and normalizing the numerical size, so as to obtain standardized image data.

[0079] When performing this step, it is necessary to save the original image obtained in step S1 and the image after segmentation preprocessing at the same time.

[0080] S3. A TCP connection is established between the client and the server, and then the image data after segmentation preprocessing is transmitted to the server; the server receives the image data and stores it in a queue. The server calls the SDK of the domestic board to take the image data out of the queue and send it to the segmentation model for inference.

[0081] The segmentation model involved in this step is a computing model based on deep learning or machine learning algorithms, which is used to process and analyze image data to identify and segment the coded characters in the image;

[0082] The training process of the segmentation model involved is as follows:

[0083] Collect images containing cable coding characters as training data;

[0084] Perform image enhancement, normalization, and data enhancement preprocessing operations on the collected training data;

[0085] According to the application scenario and requirements of the image segmentation task, select one of U-Net, FCN, SegNet, DeepLab and OCR as the basic architecture of the segmentation model;

[0086] The segmentation model of the selected infrastructure is trained using the training data after the preprocessing operation. During the training process, the model parameters are optimized by the back propagation algorithm to minimize the loss function so that the segmentation model can accurately segment the characters in the image.

[0087] Evaluate the performance of the trained segmentation model on an independent validation set and fine-tune the model based on the evaluation results;

[0088] Deploy the tuned segmentation model to actual application scenarios to achieve real-time character recognition and segmentation functions.

[0089] S4. After the server completes the reasoning, it returns the result to the client. The client then performs post-segmentation processing to extract the point set of the most appropriate contour of the character, and cuts out the area where the character is located from the image in step S1 based on the point set, specifically including:

[0090] S4.1, Filtering: The image data sent to the segmentation model includes the foreground and background. The segmentation model calculates each pixel in the image data and estimates the probability that the pixel belongs to the foreground. After all calculations are completed, the segmentation model returns a probability matrix of the same size as the image data. Then, a threshold t is set to convert the probability matrix into a binary form to obtain a binary image C after preliminary segmentation. The specific formula is as follows:

[0091] C=t*O,

[0092] Wherein, C represents the binary image after preliminary segmentation, t represents the set threshold, and O represents the image data obtained in step S1;

[0093] S4.2, dilation: select a 2x2 dilation kernel, dilate the binary image after preliminary segmentation, and connect the independent pixels in the foreground;

[0094] S4.3, determine the bounding rectangle: first use the connected component labeling algorithm CCL to identify the connected pixel area in the image, mark each connected pixel area as an independent foreground object, and calculate its minimum rectangular bounding box, then find the minimum bounding rectangle in any direction through the rotating caliper algorithm, and select the rectangle with the smallest area as the final bounding rectangle, finally determine the final bounding rectangle as the best rectangular bounding rectangle through principal component analysis, and apply it to the image data obtained in step S1;

[0095] S4.4, sorting: for each determined bounding rectangle, determine the x-coordinate of its leftmost pixel, create an array to store each bounding rectangle and its corresponding x-coordinate of the leftmost pixel, use the sorting function provided by the programming language to sort the entire array in ascending order according to the x-coordinate of the leftmost pixel, and process each bounding rectangle according to the sorting result to ensure that the bounding rectangle corresponds to its foreground object one by one;

[0096] S4.5, prevention of cross-frames: for each determined bounding rectangle, compare the y coordinates of its adjacent rectangle on the right. If the y coordinate of the rightmost pixel of one bounding rectangle is greater than the y coordinate of the leftmost pixel of its right bounding rectangle, it means that the two bounding rectangles overlap in the vertical direction. Then, move the right side of the left bounding rectangle to the right, or adjust the left side of the right bounding rectangle to the left, to ensure that the two optimal bounding rectangles do not overlap in the vertical direction. After adjustment, check each bounding rectangle again to confirm that no overlap occurs.

[0097] S4.6, image cropping: obtain the coordinates of the four vertices of each circumscribed rectangle to define the position of each character in the image data obtained in step S1, and crop the area in the image where the character is located.

[0098] S5. First, the cropped area image is rectified and pre-processed for recognition, including:

[0099] S5.1, correcting the cropped area image by perspective transformation technology;

[0100] S5.2. Scale the corrected image to the size of the segmentation model input: first, keep the height of the corrected image unchanged, and scale the width to 2 / 3 of the original width; then calculate the ratio R, which is the ratio of the width to the height in the corrected image, and the formula is as follows: R = (img.cols) / (img.rows); then, for the image after the initial scaling, determine the product of its height imgH and R and its width imgW. If the product of its height imgH and R is greater than its width imgW, the scaled width value resize_W of the image is the width imgW; if the product of its height imgH and R does not exceed its width imgW, the scaled width value resize_W of the image is the product of its height imgH and R. According to the judgment result, perform secondary scaling of the image; after completing the secondary scaling, if the image width still does not meet the input standard of the segmentation model, fill it with grayscale value;

[0101] S5.3. Convert the data type of the scaled image to a 32-bit floating point number, calculate the average of all pixel values ​​of the converted image, then divide each pixel value of the converted image by 255 and subtract the average of all pixel values, and finally divide by the variance to complete the normalization of the scaled image.

[0102] Then, step S3 is repeated to transmit the preprocessed image data to the server; the server receives the image data and stores it in a queue, and the server calls the SDK of the domestic board to take the image data out of the queue and send it to the segmentation model for reasoning.

[0103] S6. The server transmits the inference result to the client through TCP. The client compares the inference result with the preset dictionary table to obtain the recognition result.

[0104] Embodiment 2:

[0105] Combined with Figure 2 This embodiment proposes a sheath character recognition system based on a domestic environment, and its structure includes:

[0106] An image acquisition module is used to acquire images containing cable coding characters in real time through a 2D camera;

[0107] The segmentation preprocessing module is used to call the camera function through the client interface to obtain the image and perform segmentation preprocessing, including adjusting the image size, changing the data type of the pixel points and normalizing the value size, so as to obtain standardized image data;

[0108] A connection establishment module is used to establish a TCP connection between the client and the server to transmit the segmentation pre-processed image data to the server;

[0109] The data transmission module is used to receive image data from the server and store it in a queue. By calling the SDK of the domestic board, the image data is taken out of the queue and sent to the segmentation model for reasoning;

[0110] The result return module is used to return the result to the client after the segmentation model completes the inference;

[0111] The extraction and cutting module is used to perform post-segmentation processing on the client side, extract the point set of the most appropriate contour where the character is located, and cut out the area where the character is located from the image obtained by the image acquisition module according to the point set;

[0112] The correction processing module is used to correct and identify the cropped area image, and then call the connection establishment module and the data transmission module to reason about the area image through the segmentation model;

[0113] The comparison and recognition module is used to obtain the inference result of the image of the area returned to the client by the result return module, and compare the inference result with the preset dictionary table to obtain the recognition result and transmit it to the client for display.

[0114] In this embodiment, the segmentation model involved is a computing model based on a deep learning or machine learning algorithm, which is used to process and analyze image data to identify and segment the coded characters in the image.

[0115] The training process of the segmentation model involved is as follows:

[0116] Collect images containing cable coding characters as training data;

[0117] Perform image enhancement, normalization, and data enhancement preprocessing operations on the collected training data;

[0118] According to the application scenario and requirements of the image segmentation task, select one of U-Net, FCN, SegNet, DeepLab and OCR as the basic architecture of the segmentation model;

[0119] The segmentation model of the selected infrastructure is trained using the training data after the preprocessing operation. During the training process, the model parameters are optimized by the back propagation algorithm to minimize the loss function so that the segmentation model can accurately segment the characters in the image.

[0120] Evaluate the performance of the trained segmentation model on an independent validation set and fine-tune the model based on the evaluation results;

[0121] Deploy the tuned segmentation model to actual application scenarios to achieve real-time character recognition and segmentation functions.

[0122] In this embodiment, the extraction and cutting module involved performs post-segmentation processing on the client to extract the point set of the most appropriate contour of the character, and cuts out the area where the character is located from the image obtained by the image acquisition module according to the point set. The specific process includes:

[0123] Filtering: The image data sent to the segmentation model includes the foreground and background. The segmentation model calculates each pixel in the image data and estimates the probability that the pixel belongs to the foreground. After all calculations are completed, the segmentation model returns a probability matrix of the same size as the image data. Then, a threshold t is set to convert the probability matrix into a binary form to obtain a binary image C after preliminary segmentation. The specific formula is as follows:

[0124] C=t*O,

[0125] Wherein, C represents the binary image after preliminary segmentation, t represents the set threshold, and O represents the image data obtained in step S1;

[0126] Dilation: Select a 2x2 dilation kernel to dilate the binary image after preliminary segmentation to connect the independent pixels in the foreground;

[0127] Determine the bounding rectangle: first use the connected component labeling algorithm CCL to identify the connected pixel area in the image, mark each connected pixel area as an independent foreground object, and calculate its minimum rectangular bounding box, then find the minimum bounding rectangle in any direction through the rotating caliper algorithm, and select the rectangle with the smallest area as the final bounding rectangle, and finally determine the final bounding rectangle as the best rectangular bounding rectangle through principal component analysis, and apply it to the image data obtained in step S1;

[0128] Sorting: For each bounding rectangle, determine the x-coordinate of its leftmost pixel, create an array to store each bounding rectangle and its corresponding x-coordinate of the leftmost pixel, use the sorting function provided by the programming language to sort the entire array in ascending order according to the x-coordinate of the leftmost pixel, and process each bounding rectangle according to the sorting result to ensure that the bounding rectangle corresponds to its foreground object one by one;

[0129] Prevent cross-frames: For each bounding rectangle determined, compare the y coordinates of its adjacent rectangle on the right. If the y coordinate of the rightmost pixel of one bounding rectangle is greater than the y coordinate of the leftmost pixel of its right bounding rectangle, it means that the two bounding rectangles overlap in the vertical direction. Then move the right side of the left bounding rectangle to the right, or adjust the left side of the right bounding rectangle to the left to ensure that the two optimal bounding rectangles do not overlap in the vertical direction. After adjustment, check each bounding rectangle again to confirm that there is no overlap.

[0130] Image cropping: Obtain the coordinates of the four vertices of each circumscribed rectangle to define the position of each character in the image data obtained in step S1, and crop the area in the image where the character is located.

[0131] In this embodiment, the correction processing module performs correction and recognition preprocessing on the cropped area image, and the specific process includes:

[0132] Correct the cropped area image through perspective transformation technology;

[0133] Scale the corrected image to the size of the segmentation model input: first, keep the height of the corrected image unchanged, and scale the width to 2 / 3 of the original width; then calculate the ratio R, which is the ratio of the width to the height in the corrected image, and the formula is as follows: R = (img.cols) / (img.rows); then, for the image after the initial scaling, determine the product of its height imgH and R and its width imgW. If the product of its height imgH and R is greater than its width imgW, the scaled width value resize_W of the image is the width imgW; if the product of its height imgH and R does not exceed its width imgW, the scaled width value resize_W of the image is the product of its height imgH and R. According to the judgment result, perform secondary scaling of the image; after completing the secondary scaling, if the image width still does not meet the input standard of the segmentation model, fill it with grayscale value;

[0134] Convert the data type of the scaled image to a 32-bit floating point number, calculate the average of all pixel values ​​of the converted image, then divide each pixel value of the converted image by 255 and subtract the average of all pixel values, and finally divide by the variance to complete the normalization of the scaled image.

[0135] In summary, the sheath character recognition method and system based on the domestic environment of the present invention can complete the character recognition of the sheathed cable and quickly discover whether the painted characters of the sheathed cable are accurate, thereby solving the problem that the sheathed cable industry cannot observe the sprayed characters in real time with the human eye, resulting in low efficiency and high cost due to the inability to correct the spraying errors of the printer in time.

[0136] The above specific examples are used to explain the principles and implementation methods of the present invention in detail. These examples are only used to help understand the core technical content of the present invention. Based on the above specific embodiments of the present invention, any improvements and modifications made by technicians in this technical field without departing from the principles of the present invention should fall within the scope of patent protection of the present invention.

Claims

1. A sheath character recognition method based on a domestic environment, characterized in that: The steps include: S1, using a 2D camera to obtain images containing cable coding characters in real time; S2, calling the camera function through the client interface to obtain the image and perform segmentation preprocessing; S3, a TCP connection is established between the client and the server, and then the image data after segmentation preprocessing is transmitted to the server; the server receives the image data and stores it in a queue, and the server calls the SDK of the domestic board to take the image data out of the queue and send it to the segmentation model for reasoning; S4, after the server completes the reasoning, it returns the result to the client, and the client then performs post-segmentation processing to extract the point set of the most appropriate contour of the character, and cuts out the area where the character is located from the image in step S1 according to the point set; S5, correct and identify the cropped area image, then repeat step S3, and the server calls the SDK for inference; S6. The server transmits the inference result to the client through TCP. The client compares the inference result with the preset dictionary table to obtain the recognition result.

2. A method for sheath character recognition based on a domestic environment according to claim 1, characterized in that: Execute step S2 to obtain an image and perform segmentation preprocessing, specifically including adjusting the image size, changing the data type of the pixel points and normalizing the numerical values, so as to obtain standardized image data.

3. A method for sheath character recognition based on a domestic environment according to claim 1, characterized in that: The segmentation model is a computational model based on deep learning or machine learning algorithms that processes and analyzes image data to identify and segment the coded characters in the image; The training process of the segmentation model is as follows: Collect images containing cable coding characters as training data; Perform image enhancement, normalization, and data enhancement preprocessing operations on the collected training data; According to the application scenario and requirements of the image segmentation task, select one of U-Net, FCN, SegNet, DeepLab and OCR as the basic architecture of the segmentation model; The segmentation model of the selected infrastructure is trained using the training data after the preprocessing operation. During the training process, the model parameters are optimized by the back propagation algorithm to minimize the loss function so that the segmentation model can accurately segment the characters in the image. Evaluate the performance of the trained segmentation model on an independent validation set and fine-tune the model based on the evaluation results; Deploy the tuned segmentation model to actual application scenarios to achieve real-time character recognition and segmentation functions.

4. A sheath character recognition method based on a domestic environment according to claim 1, characterized in that: The step S4 specifically includes: S4.1, Filtering: The image data sent to the segmentation model includes the foreground and background. The segmentation model calculates each pixel in the image data and estimates the probability that the pixel belongs to the foreground. After all calculations are completed, the segmentation model returns a probability matrix of the same size as the image data. Then, a threshold t is set to convert the probability matrix into a binary form to obtain a binary image C after preliminary segmentation. The specific formula is as follows: C=t*O, Wherein, C represents the binary image after preliminary segmentation, t represents the set threshold, and O represents the image data obtained in step S1; S4.2, dilation: select a 2x2 dilation kernel, dilate the binary image after preliminary segmentation, and connect the independent pixels in the foreground; S4.3, determine the bounding rectangle: first use the connected component labeling algorithm CCL to identify the connected pixel area in the image, mark each connected pixel area as an independent foreground object, and calculate its minimum rectangular bounding box, then find the minimum bounding rectangle in any direction through the rotating caliper algorithm, and select the rectangle with the smallest area as the final bounding rectangle, finally determine the final bounding rectangle as the best rectangular bounding rectangle through principal component analysis, and apply it to the image data obtained in step S1; S4.4, sorting: for each determined bounding rectangle, determine the x-coordinate of its leftmost pixel, create an array to store each bounding rectangle and its corresponding x-coordinate of the leftmost pixel, use the sorting function provided by the programming language to sort the entire array in ascending order according to the x-coordinate of the leftmost pixel, and process each bounding rectangle according to the sorting result to ensure that the bounding rectangle corresponds to its foreground object one by one; S4.5, prevention of cross-frames: for each determined bounding rectangle, compare the y coordinates of its adjacent rectangle on the right. If the y coordinate of the rightmost pixel of one bounding rectangle is greater than the y coordinate of the leftmost pixel of its right bounding rectangle, it means that the two bounding rectangles overlap in the vertical direction. Then, move the right side of the left bounding rectangle to the right, or adjust the left side of the right bounding rectangle to the left, to ensure that the two optimal bounding rectangles do not overlap in the vertical direction. After adjustment, check each bounding rectangle again to confirm that no overlap occurs. S4.6, image cropping: obtain the coordinates of the four vertices of each circumscribed rectangle to define the position of each character in the image data obtained in step S1, and crop the area in the image where the character is located.

5. A method for sheath character recognition based on a domestic environment according to claim 1, characterized in that: Execute step S5 to perform correction and recognition preprocessing on the cropped area image, specifically including: S5.1, correcting the cropped area image by perspective transformation technology; S5.

2. Scale the corrected image to the size of the segmentation model input: first, keep the height of the corrected image unchanged, and scale the width to 2 / 3 of the original width; then calculate the ratio R, which is the ratio of the width to the height in the corrected image, and the formula is as follows: R = (img.cols) / (img.rows); then, for the image after the initial scaling, determine the product of its height imgH and R and its width imgW. If the product of its height imgH and R is greater than its width imgW, the scaled width value resize_W of the image is the width imgW; if the product of its height imgH and R does not exceed its width imgW, the scaled width value resize_W of the image is the product of its height imgH and R. According to the judgment result, perform secondary scaling of the image; after completing the secondary scaling, if the image width still does not meet the input standard of the segmentation model, fill it with grayscale value; S5.

3. Convert the data type of the scaled image to a 32-bit floating point number, calculate the average of all pixel values ​​of the converted image, then divide each pixel value of the converted image by 255 and subtract the average of all pixel values, and finally divide by the variance to complete the normalization of the scaled image.

6. A sheath character recognition system based on a domestic environment, characterized in that: Its structure includes: An image acquisition module is used to acquire images containing cable coding characters in real time through a 2D camera; The segmentation preprocessing module is used to call the camera function through the client interface to obtain images and perform segmentation preprocessing; A connection establishment module is used to establish a TCP connection between the client and the server to transmit the segmentation pre-processed image data to the server; The data transmission module is used to receive image data from the server and store it in a queue. By calling the SDK of the domestic board, the image data is taken out of the queue and sent to the segmentation model for reasoning; The result return module is used to return the result to the client after the segmentation model completes the inference; The extraction and cutting module is used to perform post-segmentation processing on the client side, extract the point set of the most appropriate contour where the character is located, and cut out the area where the character is located from the image obtained by the image acquisition module according to the point set; The correction processing module is used to correct and identify the cropped area image, and then call the connection establishment module and the data transmission module to reason about the area image through the segmentation model; The comparison and recognition module is used to obtain the inference result of the image of the area returned to the client by the result return module, and compare the inference result with the preset dictionary table to obtain the recognition result and transmit it to the client for display.

7. A sheath character recognition system based on a domestic environment according to claim 6, characterized in that: The segmentation preprocessing module calls the camera function through the client interface to obtain the image and perform segmentation preprocessing, which specifically includes adjusting the image size, changing the data type of the pixel points and normalizing the numerical value, so as to obtain standardized image data.

8. A sheath character recognition system based on a domestic environment according to claim 6, characterized in that: The segmentation model is a computing model based on a deep learning or machine learning algorithm, which is used to process and analyze image data to identify and segment the coded characters in the image; The training process of the segmentation model is as follows: Collect images containing cable coding characters as training data; Perform image enhancement, normalization, and data enhancement preprocessing operations on the collected training data; According to the application scenario and requirements of the image segmentation task, select one of U-Net, FCN, SegNet, DeepLab and OCR as the basic architecture of the segmentation model; The segmentation model of the selected infrastructure is trained using the training data after the preprocessing operation. During the training process, the model parameters are optimized by the back propagation algorithm to minimize the loss function so that the segmentation model can accurately segment the characters in the image. Evaluate the performance of the trained segmentation model on an independent validation set and fine-tune the model based on the evaluation results; Deploy the tuned segmentation model to actual application scenarios to achieve real-time character recognition and segmentation functions.

9. A sheath character recognition system based on a domestic environment according to claim 6, characterized in that: The extraction and cutting module performs post-segmentation processing on the client side, extracts the point set of the most appropriate contour of the character, and cuts out the area where the character is located from the image obtained by the image acquisition module according to the point set. The specific process includes: Filtering: The image data sent to the segmentation model includes the foreground and background. The segmentation model calculates each pixel in the image data and estimates the probability that the pixel belongs to the foreground. After all calculations are completed, the segmentation model returns a probability matrix of the same size as the image data. Then, a threshold t is set to convert the probability matrix into a binary form to obtain a binary image C after preliminary segmentation. The specific formula is as follows: C=t*O, Wherein, C represents the binary image after preliminary segmentation, t represents the set threshold, and O represents the image data obtained in step S1; Dilation: Select a 2x2 dilation kernel to dilate the binary image after preliminary segmentation to connect the independent pixels in the foreground; Determine the bounding rectangle: first use the connected component labeling algorithm CCL to identify the connected pixel area in the image, mark each connected pixel area as an independent foreground object, and calculate its minimum rectangular bounding box, then find the minimum bounding rectangle in any direction through the rotating caliper algorithm, and select the rectangle with the smallest area as the final bounding rectangle, and finally determine the final bounding rectangle as the best rectangular bounding rectangle through principal component analysis, and apply it to the image data obtained in step S1; Sorting: For each bounding rectangle, determine the x-coordinate of its leftmost pixel, create an array to store each bounding rectangle and its corresponding x-coordinate of the leftmost pixel, use the sorting function provided by the programming language to sort the entire array in ascending order according to the x-coordinate of the leftmost pixel, and process each bounding rectangle according to the sorting result to ensure that the bounding rectangle corresponds to its foreground object one by one; Prevent cross-frames: For each bounding rectangle determined, compare the y coordinates of its adjacent rectangle on the right. If the y coordinate of the rightmost pixel of one bounding rectangle is greater than the y coordinate of the leftmost pixel of its right bounding rectangle, it means that the two bounding rectangles overlap in the vertical direction. Then move the right side of the left bounding rectangle to the right, or adjust the left side of the right bounding rectangle to the left to ensure that the two optimal bounding rectangles do not overlap in the vertical direction. After adjustment, check each bounding rectangle again to confirm that there is no overlap. Image cropping: Obtain the coordinates of the four vertices of each circumscribed rectangle to define the position of each character in the image data obtained in step S1, and crop the area in the image where the character is located.

10. A sheath character recognition system based on a domestic environment according to claim 6, characterized in that: The correction processing module performs correction and recognition preprocessing on the cropped area image, and the specific process includes: Correct the cropped area image through perspective transformation technology; Scale the corrected image to the size of the segmentation model input: first, keep the height of the corrected image unchanged, and scale the width to 2 / 3 of the original width; then calculate the ratio R, which is the ratio of the width to the height in the corrected image, and the formula is as follows: R = (img.cols) / (img.rows); then, for the image after the initial scaling, determine the product of its height imgH and R and its width imgW. If the product of its height imgH and R is greater than its width imgW, the scaled width value resize_W of the image is the width imgW; if the product of its height imgH and R does not exceed its width imgW, the scaled width value resize_W of the image is the product of its height imgH and R. According to the judgment result, perform secondary scaling of the image; after completing the secondary scaling, if the image width still does not meet the input standard of the segmentation model, fill it with grayscale value; Convert the data type of the scaled image to a 32-bit floating point number, calculate the average of all pixel values ​​of the converted image, then divide each pixel value of the converted image by 255 and subtract the average of all pixel values, and finally divide by the variance to complete the normalization of the scaled image.

Images