A recognition method, apparatus, device and medium

Abstract

The application discloses a recognition method, device, equipment and medium, the method comprises: obtaining an image to be recognized containing at least a target object; target detection is performed on the image to be recognized to obtain a first image; wherein the first image comprises an image containing the target object segmented from the image to be recognized; direction detection is performed on the first image to obtain direction information of the first image; the first image is corrected based on the direction information to obtain a second image; the second image is recognized to obtain a recognition result of the target object.

Description

Technical Field

[0001] This application relates to the field of image recognition technology, and in particular to a recognition method, apparatus, device and medium. Background Technology

[0002] In practical applications, the recognition of sign content is usually achieved through manual recognition or optical character recognition (OCR). However, manual recognition is inefficient, and OCR cannot be implemented when the sign position or the acquisition angle is uncertain. Summary of the Invention

[0003] In view of the above problems, this application provides an identification method, apparatus, device and medium.

[0004] The technical solution provided in this application is as follows:

[0005] This application provides an identification method, the method comprising:

[0006] Obtain an image to be identified that contains at least the target object;

[0007] Target detection is performed on the image to be identified to obtain a first image; wherein, the first image includes an image containing the target object obtained by segmenting from the image to be identified;

[0008] The first image is subjected to orientation detection to obtain the orientation information of the first image;

[0009] Based on the direction information, the first image is subjected to orientation correction processing to obtain the second image;

[0010] The second image is then used for recognition to obtain the recognition result of the target object.

[0011] In one implementation, the step of performing orientation correction processing on the first image based on the orientation information to obtain the second image includes:

[0012] Determine the first contour information of a first graphic that circumvents the edge of the first image; wherein, the first graphic includes a closed graphic composed of multiple straight lines connected end to end in sequence;

[0013] Based on the direction information, the orientation correction process is performed on the corner position information in the first contour information to obtain the second image.

[0014] In one implementation, the first image includes a binary segmented image containing the target object obtained by segmenting the image to be identified; the step of performing target detection on the image to be identified to obtain the first image includes:

[0015] The target detection image is obtained by performing target detection on the image to be identified through the feature extraction module in the trained target detection network; wherein, the feature extraction module includes MobilenetV2; the feature extraction module includes multiple feature extraction units.

[0016] In one implementation, the step of performing orientation detection on the first image to obtain orientation information of the first image includes:

[0017] The feature fusion module in the trained target detection network performs feature fusion on the feature extraction results output by at least two feature extraction units of the trained feature extraction module and the binary segmentation image to obtain the orientation information of the first image.

[0018] In one embodiment, the method further includes:

[0019] Obtain sample data containing annotation information; wherein, the sample data includes image data containing a target object; the annotation information includes the position information of the target object in the image data and the orientation information of the image region containing the target object;

[0020] The feature extraction module performs target detection on the sample data to obtain a binary segmentation image corresponding to the sample data.

[0021] The feature fusion module performs feature fusion on the binary segmented image corresponding to the sample data and the feature extraction results output by at least two feature extraction units of the feature extraction module to obtain the orientation information corresponding to the sample data.

[0022] Based on the annotation information, the binary segmentation image corresponding to the sample data, and the orientation information corresponding to the sample data, the parameters of the feature extraction module and the feature fusion module are adjusted to obtain the trained target detection network.

[0023] In one embodiment, the step of recognizing the second image to obtain a recognition result of the target object includes:

[0024] The second image is subjected to deformation correction to obtain a deformation-corrected image;

[0025] The deformation-corrected image is identified to obtain the identification result.

[0026] In one embodiment, performing deformation correction on the second image to obtain a deformation-corrected image includes:

[0027] Based on the edge position information of the second image, the second contour information of the second graphic circumferential to the edge position information is determined; wherein, the second graphic includes a closed graphic composed of multiple straight lines connected end to end in sequence;

[0028] Based on the edge position information and the second contour information, the second image is deformed to obtain the deformed image.

[0029] In one implementation, the deformation correction of the second image based on the edge location information and the second contour information includes:

[0030] Obtain the degree of fit between the edge position information and the second contour information;

[0031] If the fitting degree information is greater than the fitting threshold, the second image is deformed by Bézier curve fitting.

[0032] If the fitting degree information is less than or equal to the fitting threshold, the second image is deformed by transmission transformation.

[0033] This application embodiment also provides an identification device, the device comprising:

[0034] The acquisition module is used to acquire an image to be identified that contains at least the target object;

[0035] The detection module is used to perform target detection on the image to be identified to obtain a first image; and to perform orientation detection on the first image to obtain orientation information of the first image; wherein, the first image includes an image containing the target object segmented from the image to be identified;

[0036] The processing module is used to perform orientation correction processing on the first image based on the orientation information to obtain the second image;

[0037] The recognition module is used to recognize the second image and obtain the recognition result of the target object.

[0038] This application also provides an identification device, which includes a processor and a memory; wherein the memory stores a computer program, and when the computer program is executed by the processor, it can implement the identification method as described above.

[0039] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor of an electronic device, can implement the identification method described above.

[0040] As can be seen from the above, the recognition method provided in this application performs target detection on the image to be recognized, which contains at least the target object, to obtain a first image containing the target object segmented from the image to be recognized. Then, the first image is subjected to orientation detection to obtain the orientation information of the first image. Based on the orientation information, the first image is subjected to orientation correction processing to obtain a second image. The second image is then recognized to obtain the recognition result of the target object.

[0041] Therefore, the recognition method provided in this application performs target detection and segmentation on the image to be recognized to obtain a first image. This not only reduces interference from the background image but also reduces the number of pixels in the first image, reducing the computational load of subsequent direction detection and recognition processing. It also accurately determines the position information of the target object in the image to be recognized. Furthermore, after performing direction detection on the first image to obtain direction information, and then performing orientation correction processing on the first image based on the direction information to obtain a second image, redundant orientation information in the second image is removed compared to the first image. In other words, the second image only contains image data associated with the target object in the image to be recognized, and carries less redundant orientation information, thereby reducing the interference of redundant orientation information on the recognition of the second image and further improving the efficiency and accuracy of image recognition.

[0042] When the target object is a sign, the identification method provided in this application can overcome the technical problem in related technologies that the sign cannot be accurately identified or even identified at all due to the uncertain position of the sign. This achieves automated, high-precision, and real-time identification of the sign, thereby improving the automation and intelligence level of sign acceptance. Attached Figure Description

[0043] Figure 1 A flowchart illustrating the identification method provided in this application embodiment;

[0044] Figure 2 A schematic diagram illustrating the process of obtaining a second image by performing orientation correction on a first image, as provided in an embodiment of this application;

[0045] Figure 3A This is a structural schematic diagram of the first image and first contour information provided in the embodiments of this application;

[0046] Figure 3B This is a schematic diagram of the structure of the second image provided in the embodiments of this application;

[0047] Figure 4A This is a schematic diagram of the target detection network provided in an embodiment of this application;

[0048] Figure 4B A schematic diagram of the structure of the first fusion unit provided in an embodiment of this application;

[0049] Figure 5 A schematic diagram illustrating the training process of the target detection network provided in this application embodiment;

[0050] Figure 6 A flowchart illustrating the training of the target detection model is provided for embodiments of this application;

[0051] Figure 7 A schematic diagram illustrating the process of recognizing a second image provided in an embodiment of this application;

[0052] Figure 8 This is a functional structure diagram of the model training module provided in an embodiment of this application;

[0053] Figure 9A This is a schematic diagram of the first process for performing deformation correction on a second image according to an embodiment of this application.

[0054] Figure 9B This is a schematic diagram of a second process for performing deformation correction on a second image, provided in an embodiment of this application.

[0055] Figure 10 This is a schematic diagram of the structure of the identification device provided in the embodiments of this application;

[0056] Figure 11 This is a schematic diagram of the structure of the identification device provided in an embodiment of this application. Detailed Implementation

[0057] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings.

[0058] It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit this application.

[0059] In practical applications, signs are generally regular geometric shapes, such as rectangles or rectangular-like shapes with rounded corners. They typically include text or graphics to describe or label the items they represent, such as equipment attribute labels in a computer room or price tags in a store. After the sign design and production are completed, the content usually needs to be identified or verified. Related technologies typically involve manual verification or OCR methods to identify the sign content and determine its compliance based on the verification results.

[0060] For example, in the construction of a computer room, equipment labels are the most common tool for marking equipment. The same type of equipment usually needs to be labeled according to a predetermined format. In the equipment acceptance process after the completion of the computer room construction, the workload of equipment label acceptance is large, and it is mainly done manually. Such acceptance method is inefficient and costly.

[0061] Furthermore, the process of verifying the compliance of signs is mainly done manually in the backend. After the front-end engineering construction personnel submit images containing signs, the backend staff also need to confirm whether the image collection time, location, sign placement method, and content all meet the requirements. This manual processing method further prolongs the sign recognition and acceptance time.

[0062] However, manual identification and confirmation methods are inefficient and costly. While OCR can improve the efficiency of sign recognition and reduce the cost to some extent, it cannot accurately identify and confirm signs when the sign's position or shooting angle is uncertain.

[0063] To address the above issues, embodiments of this application provide an identification method, apparatus, device, and medium. The identification method provided in this application performs target detection on an image to be identified, containing at least a target object, to obtain a first image segmented from the image to be identified, containing the target object. This reduces the probability of identification failure due to uncertainties in the target object's position and the shooting angle. After performing orientation detection on the first image to obtain its orientation information, the method performs orientation correction processing on the first image based on the orientation information to obtain a second image. The second image is then identified to obtain the identification result of the target object. Thus, when the orientation of the second image is regular, such as horizontal or vertical, the computational load during second image identification is reduced, thereby improving the efficiency of second image identification.

[0064] When the target object is a sign and the image to be identified is an image containing the sign, the identification method provided in this application embodiment can accurately segment the image containing the sign, thereby reducing the number of pixels in the first image and reducing the influence of background pixels in the image to be identified on the target object. After obtaining the orientation information of the first image, the second image obtained by performing orientation correction on the first image based on the orientation information has orientation information that meets the identification conditions, thereby reducing the amount of computation required for identification and further improving the efficiency and accuracy of identification, thus realizing automated, high-precision, and real-time identification of signs.

[0065] It should be noted that the identification method provided in this application embodiment can be implemented by the processor of the identification device. The processor is at least one of the following: Application Specific Integrated Circuit (ASIC), Digital Signal Processor (DSP), Digital Signal Processing Device (DSPD), Programmable Logic Device (PLD), Field Programmable Gate Array (FPGA), Central Processing Unit (CPU), Controller, Microcontroller, and Microprocessor.

[0066] For example, the identification device may be a thin client, a thick client, a handheld or laptop device, a microprocessor-based system, a set-top box, a programmable consumer electronics product, a networked personal computer, a minicomputer system, etc. For example, the identification device may integrate an image acquisition device.

[0067] Identification devices can execute instructions through program modules. Typically, program modules can include routines, programs, object programs, components, logic, data structures, etc., which perform specific tasks or implement specific abstract data types. Computer systems / servers can be implemented in distributed cloud computing environments, where tasks are executed by remote processing devices linked via communication networks. In distributed cloud computing environments, program modules can reside on local or remote computing system storage media, including storage devices.

[0068] Figure 1 A flowchart illustrating the identification method provided in the embodiments of this application is shown below. Figure 1 As shown, the method may include steps 101 to 105:

[0069] Step 101: Obtain an image to be identified that contains at least the target object.

[0070] In one embodiment, the image to be identified may be obtained by the identification device from other devices or from the network via a network connection; for example, the image to be identified may also be stored in the storage space of the identification device; for example, the image to be identified may also be acquired in real time by an image acquisition device set in the image to be identified.

[0071] In one implementation, the target object may be a sign containing text and / or patterns; for example, the number of signs contained in the image to be recognized may be multiple; for example, the sign may include any one of a product price tag, a product ingredient label, and a device attribute label.

[0072] In one implementation, the image to be identified may also include image data containing the environment in which the target object is located, i.e., background image data. For example, if the target object is a product label, the image to be identified may include image data of the container where the product label is set.

[0073] Step 102: Perform target detection on the image to be recognized to obtain the first image.

[0074] The first image includes an image containing the target object, segmented from the image to be identified.

[0075] In one implementation, the first image may be an image with the same area as the image to be identified, in which case the target object may occupy the entire image area of ​​the image to be identified; for example, the first image may be an image with an area smaller than the image to be identified, in which case the image area where the target object is located may occupy a portion of the image area of ​​the image to be identified.

[0076] In one implementation, target detection is performed on the image to be recognized to obtain a first image, which can be achieved in any of the following ways:

[0077] The system detects whether the image to be identified contains an image region with a regular geometric shape, segments the image region from the image to be identified, and determines the segmented image data as the first image. For example, the regular geometric shape can be the same as the geometric shape of a sign, such as a square or rectangle.

[0078] Based on threshold segmentation or edge segmentation methods, target detection and image segmentation are performed on the image to be recognized to obtain the first image.

[0079] Step 103: Perform orientation detection on the first image to obtain the orientation information of the first image.

[0080] In one implementation, the orientation information of the first image can be represented by whether the geometric centerline of the first image is consistent with the target orientation; for example, the target orientation may include any orientation between the horizontal and vertical directions.

[0081] In one implementation, the orientation information of the first image can be represented by the angle between the geometric center line of the first image and the target direction; for example, when the target direction is horizontal, the orientation information of the first image can be an angle of 30 degrees with the horizontal direction.

[0082] In one implementation, the orientation information of the first image can be represented by numbering the angle intervals. For example, the 0-360 degree range can be divided into 8 angle intervals, each containing 45 degrees. For instance, the first angle interval is [0, 45], the second angle interval is [46, 90], and so on. If the orientation information of the first image is 75 degrees, it can be represented as the second angle interval. To simplify the representation, the orientation information of the first image can be represented by 2. For instance, if the first angle interval is represented by 1 and the eighth angle interval by 8, the value range of the orientation information of the first image can be [1, 8]. For instance, if the first angle interval is represented by 0 and the eighth angle interval by 7, the value range of the orientation information of the first image can be [0, 7].

[0083] Step 104: Perform orientation correction processing on the first image based on the orientation information to obtain the second image.

[0084] In one implementation, the second image is obtained by performing orientation correction processing on the first image based on orientation information, which can be achieved in any of the following ways:

[0085] Based on the orientation information, the first image is rotated until the geometric midline of the first image is the same as or opposite to the target orientation, and the rotated image is used as the second image.

[0086] Based on the orientation information, the first image is rotated until the angle between the geometric midline of the first image and the target direction is the target angle, and the rotated image is used as the second image; for example, the target angle can be 90 degrees.

[0087] Step 105: Recognize the second image to obtain the recognition result of the target object.

[0088] In one implementation, the identification result of the target object may include text data carried by the target object; for example, when the target object is a sign, the identification result of the sign may include at least one of the following: price, ingredient description, attribute description, product number, and production date.

[0089] In one embodiment, the identification result of the target object may include pattern data carried by the target object; for example, when the target object is a sign, the identification result of the sign may include at least one of the following data associated with the trademark: patterns, graphics, designs, and lines.

[0090] In one implementation, the identification result of the target object may include whether the target object is a specified object; for example, if the target object is a specified object, the identification result may indicate that the target object is a compliant object; for example, if the target object is not a specified object, the identification result may indicate that the target object is a non-compliant object.

[0091] In one implementation, the identification result of the target object may include whether the target object is an object of a specified type; for example, the identification result may include whether the target object is a price tag.

[0092] As can be seen from the above, the recognition method provided in this application performs target detection on the image to be recognized, which contains at least the target object, to obtain a first image containing the target object segmented from the image to be recognized. Then, the first image is subjected to orientation detection to obtain the orientation information of the first image. Based on the orientation information, the first image is subjected to orientation correction processing to obtain a second image. Only then is the second image recognized to obtain the recognition result of the target object.

[0093] Therefore, the recognition method provided in this application performs target detection and segmentation on the image to be recognized to obtain a first image, thereby reducing the number of pixels in the first image, weakening the influence of background image data on the target object, reducing the computational load of subsequent direction detection and recognition processing, and accurately determining the position information of the target object in the image to be recognized. Furthermore, after obtaining direction information by direction detection of the first image, and then performing orientation correction processing on the first image based on the direction information to obtain a second image, the redundant orientation information in the second image is removed compared to the first image. In other words, the second image only contains image data associated with the target object in the image to be recognized, and carries less redundant orientation information, thereby reducing the interference of redundant orientation information on the recognition of the second image, effectively reducing the computational load of recognition, and further improving the efficiency and accuracy of image recognition. When the target object is a sign, the recognition method provided in this application can overcome the technical problem in related technologies where the sign's position is uncertain, making it impossible to accurately identify the sign or even identify it at all. This achieves automated, high-precision, and real-time recognition of signs, thereby improving the automation and intelligence level of sign acceptance.

[0094] Based on the foregoing embodiments, the recognition method provided in this application performs orientation correction processing on the first image based on direction information to obtain the second image, which can be obtained through... Figure 2 The process shown is implemented. Figure 2 This is a schematic diagram illustrating the process of obtaining a second image by performing orientation correction on a first image, as provided in an embodiment of this application. Figure 2 As shown, the process may include steps A1 to A2:

[0095] Step A1: Determine the first contour information of the first graphic that circumscribes the edge of the first image.

[0096] The first figure includes a closed figure composed of multiple straight lines connected end to end in sequence.

[0097] In one implementation, the first shape may be at least one of a triangle, a rectangle, and a square; for example, the first shape may be a regular planar geometric shape.

[0098] In one implementation, the first graphic may be a graphic that contains all the pixels of the first image.

[0099] In one implementation, the first contour information may include pixel position information of each edge of the first graphic.

[0100] In one implementation, the first shape can be determined in the following way:

[0101] Obtain the contour information of the first image, determine the tangents corresponding to the contour information of the first image according to the extension direction of the first image, extend each tangent, and when each tangent intersects, determine the first shape based on the corner points where each tangent intersects and each tangent; for example, each tangent and the corner points where each tangent intersects can be determined as the first contour information.

[0102] Figure 3A This is a structural schematic diagram of the first image and first contour information provided in an embodiment of this application. (See attached diagram.) Figure 3A As shown, the first image can be a rectangular area with a certain width and extending in any direction, occupied by the grid pattern in the image. The edges of the first image can be irregular closed curves 301. Correspondingly, the first contour information 302 can be a rectangle containing all the pixels of the first image, and the intersection of each edge of the first contour information 302 yields a first corner point 3021 (x0, y0), a second corner point 3022 (x1, y1), a third corner point 3023 (x2, y2), and a fourth corner point 3024 (x3, y3). For example, the angle between any direction and the horizontal direction can be 30 degrees.

[0103] Step A2: Based on the orientation information, perform orientation correction processing on the corner position information in the first contour information to obtain the second image.

[0104] For example, since the orientation information of the first image can be arbitrary, the orientation of the first contour information can also be arbitrary. Therefore, based on the orientation information, the starting corner point can be determined from multiple corner points of the first contour information, and each corner point can be sorted in turn. Then, based on the orientation information, the sorted corner points and the edges between adjacent corner points in the first contour information can be rotated to achieve the overall rotation of the first image and obtain the second image. In this way, an image containing only the target object and with single orientation information can be obtained.

[0105] Figure 3B This is a structural schematic diagram of the second image provided in an embodiment of this application. (See attached diagram.) Figure 3B As shown, the orientation information of the second image obtained by correcting the corner positions in the first contour information based on orientation information can be in the horizontal direction, meaning that the second image no longer carries redundant orientation information.

[0106] As can be seen from the above, in the recognition method provided in this application embodiment, after obtaining the direction information of the first image, the first contour information of the first graphic adjacent to the edge of the first image can be determined, and the corner position information of the first contour information can be oriented and corrected based on the direction information to obtain the second image. Since the first graphic is a closed graphic composed of multiple straight lines connected end to end, the oriented correction processing of the corner position information in the first contour information based on the direction information can reduce the probability of pixel omission in the oriented correction of the first image, and also improve the efficiency of oriented processing of the first image.

[0107] Based on the foregoing embodiments, in the recognition method provided in this application, the first image includes a binary segmented image containing the target object obtained by segmenting the image to be recognized.

[0108] For example, the first image 301 can be as follows: Figure 3A The pixel data shown in the grid-like region; for example, in practical applications, the first image can be displayed in a specified color such as black, while the other pixel data in the image to be identified, excluding the first image, can be displayed in white. In this way, by segmenting the black region in the image to be identified after target detection, the first image can be obtained.

[0109] As can be seen from the above, in the recognition method provided in this application embodiment, the first image is a binary segmentation image. That is, when the first image is obtained after target detection of the image to be recognized, accurate segmentation between the background and the target object in the image to be recognized is achieved. Furthermore, the orientation correction of the first image based on orientation information not only eliminates the need for training a text orientation detection model in subsequent recognition operations but also improves the performance of target object recognition, such as text objects. At the same time, for cases where the image to be recognized contains multiple target objects, the binary segmentation image can also effectively reduce the computational workload of determining the attribution relationship between text and signs.

[0110] In this embodiment of the application, target detection is performed on the image to be recognized to obtain a first image, which can be achieved in the following ways:

[0111] The feature extraction module in the trained target detection network performs target detection on the image to be recognized, resulting in a binary segmentation image.

[0112] The feature extraction module includes MobilenetV2 and comprises multiple feature extraction units.

[0113] For example, the trained object detection network may include a feature extraction module and a feature fusion module, wherein the feature fusion module is used to fuse the feature information extracted by the feature extraction module.

[0114] In one implementation, when the feature extraction module is MobilenetV2, the feature extraction unit of the feature extraction module can be a linear bottleneck layer of MobilenetV2 (Linear BottleNeck). For example, MobilenetV2 can include multiple Linear BottleNecks and inverted residual network structures associated with multiple Linear BottleNecks. For example, the stride of the depthwise convolution of the Linear BottleNeck can be 2.

[0115] For example, the binary segmentation image can be a single-channel image, where the pixel value of each pixel represents whether that location is a target object, such as a sign. For example, it can be determined whether the pixel value of a certain pixel in the image to be identified is greater than a pixel threshold. If it is greater than the pixel threshold, the pixel value can be set to 1 to indicate that the pixel is a pixel in the text image; if the pixel value is less than the pixel threshold, the pixel value can be set to 0. For example, the pixel threshold can be any value between 0 and 1. For example, the larger the pixel threshold value, the lower the recall rate. The connected regions in the binarized image can be identified as the first image containing the target object.

[0116] MobileNetV2 is essentially a lightweight convolutional neural network. Since the linear bottleneck structure of MobileNetV2 is essentially a 1x1 convolutional layer without ReLU, the probability of feature information being destroyed can be reduced through the linear bottleneck structure. Furthermore, although the number of channels at the data input and output ports of the linear bottleneck structure is relatively small, the number of channels can be increased inside the linear bottleneck structure, thereby increasing the feature mapping space and improving the feature extraction effect. Due to the existence of the inverse residual network structure of MobileNetV2, the gradient vanishing problem in the feature extraction process of neural networks can also be overcome.

[0117] Based on the above advantages, using MobilenetV2 for target detection in the image to be recognized can not only reduce the computational load of target detection, but also improve the accuracy of target detection and image segmentation.

[0118] As can be seen from the above, in the recognition method provided in this application embodiment, the feature extraction module including MobilenetV2 is used to perform target detection on the image to be recognized. With the help of the excellent network structure and efficient feature extraction characteristics of MobilenetV2, not only can the efficiency of target detection on the image to be recognized be improved, but also the accuracy of target detection can be improved, thus laying the foundation for subsequent high-precision recognition.

[0119] Based on the foregoing embodiments, the recognition method provided in this application, which performs orientation detection on the first image to obtain the orientation information of the first image, can be implemented in the following ways:

[0120] The feature fusion module in the trained target detection network fuses the feature extraction results output by at least two feature extraction units of the feature extraction module and the binary segmented image to obtain the orientation information of the first image.

[0121] In one implementation, the feature fusion module may also include multiple feature fusion units. The input data of the k-th feature fusion unit may include the feature extraction result output by the m-th feature extraction unit and the output data of the (k-1)-th feature fusion unit. For example, the input data of the first feature fusion unit may include the binary segmented image and the feature extraction result output by the (M-1)-th feature extraction unit. Here, k and m are integers greater than 1, and M is an integer greater than m.

[0122] Since the network structure of the feature extraction module and the feature fusion module in the trained object detection network, as well as the way they are combined, are the same as those of the feature extraction module and the feature fusion module in the object detection network, the structure of the feature extraction module and the feature fusion module will be explained below using the network structure of the object detection network as an example.

[0123] Figure 4A This is a schematic diagram of the target detection network 4 provided in an embodiment of this application. Figure 4A As shown, the target detection network may include a feature extraction module 401 and a feature fusion module 402; wherein, the feature extraction module 401 may be MobileNetV2, and the feature extraction unit of the feature extraction module 401 may be a linear bottleneck structure; the feature fusion module 402 may include a first fusion unit 4021, a second fusion unit 4022, a third fusion unit 4023, a fourth fusion unit 4024, a fifth fusion unit 4025, a sixth fusion unit 4026, and a seventh fusion unit 4027; for example, the structures of each fusion unit may be different.

[0124] For example, the first fusion unit 4021, the second fusion unit 4022, and the third fusion unit 4023, which are directly connected to the feature extraction unit of the feature extraction module 401, can have the same structure. The structure of the first to third fusion units will be described below using the structure of the first fusion unit 4021 as an example.

[0125] Figure 4B This is a schematic diagram of the structure of the first fusion unit 4021 provided in an embodiment of this application. Figure 4B As shown, the first fusion unit 4021 may include a first convolutional subunit 40211, a second convolutional subunit 40212, a feature fusion subunit 40213, and an upper pooling subunit 40214. The convolutional kernel of the first convolutional subunit 40211 can be 3x3, the convolutional kernel of the second convolutional subunit 40212 can be 1x1, and the feature fusion subunit 40213 can implement the common concat operation in neural networks. The upper pooling subunit 40214 can receive a binary segmented image, perform upper pooling on it to obtain a first upper pooling result, and then send it to the feature fusion subunit 40213. The feature fusion subunit 40213 can also receive the m-th feature extraction result output by the m-th feature extraction unit. The feature fusion subunit 40213 performs feature fusion on the m-th feature extraction result and the first upper pooling result to obtain a first fusion result. The first fusion unit 4021 can further perform convolution processing on the first fusion result through the second convolution subunit 40212 and the first convolution subunit 40211 to further extract the feature information of the target object in the image to be identified and obtain a first feature map.

[0126] For example, the pooling unit of the nth fusion module can receive the (n-1)th result output by the (n-1)th fusion module and perform a pooling operation on it to obtain the nth pooling result. The pooling unit of the nth fusion module can perform a feature fusion operation on the nth pooling result and the mth feature extraction result output by the mth feature extraction unit. Here, n is an integer greater than 1 and less than 3.

[0127] For example, the input data of the pooling subunit 40211 may include a binary segmented image with a size of 1 / 32 of the size of the image to be recognized and 320 channels; the input data of the feature fusion subunit 40212 may include the M-1th feature extraction result with a size of 1 / 16 of the size of the image to be recognized and 96 channels; the input data of the feature fusion subunit of the second fusion unit 4022 may include the feature extraction result with a size of 1 / 8 of the size of the image to be recognized and 32 channels; and the input data of the feature fusion subunit of the third fusion unit 4023 may include the feature extraction result with a size of 1 / 4 of the size of the image to be recognized and 24 channels.

[0128] For example, the size of the first feature map can be 1 / 16 of the size of the image to be recognized, and the number of channels can be 128; the second feature map can be the feature fusion result output by the second fusion unit 4022, and its size can be 1 / 8 of the size of the image to be recognized, and the number of channels can be 64; the third feature map can be the feature fusion result output by the third fusion unit 4023, and its size can be 1 / 4 of the size of the image to be recognized, and the number of channels can be 32.

[0129] For example, the fourth fusion unit 4024 can perform a convolution operation on the third feature map output by the third fusion unit 4023 using, for example, a 3x3 convolution kernel to obtain a fourth feature map. The size of the fourth feature map can be 1 / 4 of the size of the image to be recognized, and the number of channels can be 32. For example, the fifth fusion unit 4025 can perform a point convolution operation on the fourth feature map using, for example, a 1x1 convolution kernel to obtain a fifth feature map. The size of the fifth feature map can be 1 / 4 of the size of the image to be recognized, and the number of channels can be 1. For example, the input data of the sixth fusion unit 4026 can include the fourth feature map and the fifth feature map. It is used to perform Region of Interest Pooling (ROI Pooling) on ​​the fourth feature map and the fifth feature map, thereby pooling the fourth feature map and the fifth feature map of different sizes into a sixth feature map of fixed size, and inputting it into the seventh fusion unit 4027. For example, the sixth fusion unit 4026 can divide the fifth feature map into 7x7 image blocks.

[0130] For example, the seventh fusion unit 4027 includes a third convolution subunit 40271 and a fourth convolution subunit 40272. For example, the third convolution subunit 40271 can have 1024 channels and is used to perform a full convolution operation on the sixth feature map to obtain the first result. The fourth convolution subunit 40272 can have 8 channels and can perform a full convolution operation on the first result to classify the orientation of the first image, thereby obtaining the orientation information of the first image.

[0131] As can be seen from the above, in the recognition method provided in this application embodiment, the feature fusion results of each feature fusion unit in the feature fusion module of the trained target detection network are processed by ROI Pooling, which can unify feature maps of different sizes into feature maps of the same size. This not only reduces the amount of computation required to determine the orientation information, but also improves the accuracy of the orientation information.

[0132] Based on the foregoing embodiments, the identification method provided in this application may further include... Figure 5 The steps are shown. Figure 5 This document provides a flowchart illustrating the training process of an object detection network for embodiments of this application. Figure 5 As shown, the process may include steps B1 to B4:

[0133] Step B1: Obtain sample data containing annotation information.

[0134] The sample data includes image data containing the target object and annotation information, including the location information of the target object in the image data and the orientation information of the image region containing the target object.

[0135] In one implementation, the sample data may include image data containing multiple target objects.

[0136] In one implementation, the sample data may include image data containing the target object acquired by an image acquisition device; for example, the image acquisition device may be a camera; for example, the image acquisition device may be installed in other electronic devices, such as the camera installed in a smartphone; for example, the data acquired by the image acquisition device may be cleaned first.

[0137] In one embodiment, the sample data may further include image data generated by performing a sample augmentation operation on image data acquired by the image acquisition device. For example, the sample augmentation operation may include any of the following operations on the data acquired by the image acquisition device: translation, rotation, transmission transformation, brightness adjustment, and contrast adjustment, to expand the diversity of the sample data.

[0138] In one implementation, the sample data may further include images generated based on preset signage requirements, and these images are set in the acquired scene background to expand the sample diversity.

[0139] In one implementation, at least one of the following in each sample data set may be different: the color of the target object, the background color, the text font size, the text format, and the pattern style; for example, the shooting angle of each sample data set may also be different.

[0140] In one implementation, the position information of the target object in the image data can be represented by the pixel position of the target object in the image data; the orientation information of the image region containing the target object can include at least one of whether the image region where the target object is located is tilted in the sample data, the tilt direction, and the tilt angle.

[0141] In one implementation, the labeling information may also include the content of the target object, such as text information, patterns, labels, and the combination of text and patterns.

[0142] In one implementation, the annotation information of the sample data can be determined by manual annotation, but this application does not limit this method.

[0143] Step B2: Perform target detection on the sample data using the feature extraction module to obtain a binary segmentation image corresponding to the sample data.

[0144] In one implementation, the feature extraction module may have initial feature extraction parameters, and perform target detection on the sample data based on the initial feature extraction parameters to obtain a binary segmentation map corresponding to the sample data.

[0145] Step B3: The feature fusion module performs feature fusion on the binary segmented image corresponding to the sample data and the feature extraction results output by at least two feature extraction units of the feature extraction module to obtain the orientation information corresponding to the sample data.

[0146] In one implementation, each feature fusion unit of the feature fusion module may have initial feature fusion parameters, and based on the initial feature fusion parameters, feature fusion is performed on the feature extraction results output by at least two feature extraction units of the feature extraction module and the binary segmentation image to obtain directional information corresponding to the sample data.

[0147] Step B4: Based on the annotation information, the binary segmentation image corresponding to the sample data, and the orientation information corresponding to the sample data, adjust the parameters of the feature extraction module and the feature fusion module to obtain the trained target detection network.

[0148] In one implementation, the parameters of the feature extraction module and the feature fusion module can be determined based on the first degree of matching between the binary segmented image corresponding to the sample data and its positional information in the annotation information, and the second degree of matching between the directional information corresponding to the sample data and its directional information in the annotation information.

[0149] In one implementation, a first threshold and a second threshold can be preset. If the first matching degree is less than or equal to the first threshold and the second matching degree is less than or equal to the second threshold, the network parameter adjustment operation of the feature extraction module and the feature fusion module is stopped, and the trained feature extraction module and the trained feature fusion module are obtained, thereby obtaining the trained target detection network. If the first matching degree is greater than the first threshold or the second matching degree is greater than the second threshold, the parameter adjustment operation of the feature extraction module and the feature fusion module based on sample data continues until the first matching degree is less than or equal to the first threshold and the second matching degree is less than or equal to the second threshold.

[0150] As can be seen from the above, since the sample data carries annotation information, the training process of the feature extraction module and the feature fusion module based on the sample data can be effectively shortened, and the accuracy of feature extraction and feature fusion of the feature extraction module and the feature fusion module can be improved, thereby improving the feature extraction efficiency and accuracy of the trained target detection network.

[0151] Figure 6 This document provides a flowchart illustrating the training process for an object detection model, as shown in the embodiments of this application. Figure 6 As shown, the process may include steps 601 to 603:

[0152] Step 601: Obtain image data.

[0153] For example, the image data here can be data acquired by the image acquisition device in the foregoing embodiments, as well as automatically generated image data.

[0154] Step 602: Perform sample augmentation on the image data to obtain sample data.

[0155] For example, sample augmentation of image data can be achieved through the sample augmentation operation in the foregoing embodiments.

[0156] Step 603: Train the target detection model based on the sample data.

[0157] For example, the target detection model may include the target detection network 4, which includes a feature extraction module 401 and a feature fusion module 402, as described in the foregoing embodiments.

[0158] As can be seen from the above, by augmenting the image data, the diversity of the sample data can be expanded, thereby making the target detection model trained based on the sample data more robust.

[0159] Based on the foregoing embodiments, the recognition method provided in this application recognizes the second image to obtain the recognition result of the target object, which can be achieved through... Figure 7 The process shown is implemented as follows: Figure 7 This is a schematic diagram of the process for recognizing a second image provided in an embodiment of this application, such as... Figure 7 As shown, the process may include steps C1 to C2:

[0160] Step C1: Perform deformation correction on the second image to obtain a deformation-corrected image.

[0161] In one implementation, when acquiring the image to be identified, the target object, such as a sign, may be bent, worn, or wrinkled. There may also be cases where the acquisition angle of the image acquisition device is not the specified angle. As a result, the target object in the first image of the image to be identified and the second image obtained by azimuth correction of the first image may have different degrees of deformation, such as the sign not being coplanar. Therefore, deformation correction is required for the second image.

[0162] In one embodiment, deformation correction of the second image may include processing the second image using methods such as pixel smoothing and pixel interpolation to achieve deformation correction of the second image.

[0163] In one implementation, the deformation-corrected image may be the same size as the second image.

[0164] Step C2: Recognize the deformation-corrected image to obtain the recognition result.

[0165] For example, the recognition of deformed images can also be achieved through a trained recognition network; for example, the recognition network can perform both text detection and text recognition functions.

[0166] For example, text detection can be implemented using an Efficient and Accurate Scene Text Detector (EAST). EAST can be based on MobileNetV2 as its network, and its structure can be... Figure 4AAs shown, the sample data collected during the training process can be image data of size 512x512; for example, text recognition can use a convolutional recurrent neural network (CRNN), the structure of which can be as follows. Figure 4A As shown, its base network can be ResNet50, and the sample data used in its training process can be image data with a size of 32x100. The text category can be 5990, and the sample data can contain at least one character data among Chinese characters, English words, English letters, and numbers.

[0167] For example, recognizing a deformed image using a trained recognition network can include text detection, text recognition, and text feature extraction to obtain a text recognition result. For example, the trained recognition network can also determine whether a target object is compliant by judging the text recognition result. For example, the trained recognition network can judge the text recognition result based on preset rules, such as using a text regular expression matching strategy. For example, after determining whether the target object is compliant, a prompt message indicating whether the target object is compliant can be output.

[0168] As can be seen from the above, in the recognition method provided in the embodiments of this application, after obtaining the second image, the second image is first deformed to obtain a deformed image, and then the deformed image is recognized to obtain the recognition result, thereby reducing the interference of the deformation of the target object on the recognition process and further improving the accuracy of the recognition result.

[0169] In this embodiment of the application, the trained target detection network and the trained recognition network can be obtained by training the target detection network and the recognition network through the model training module. After each network is trained, the model training module can also set each trained neural network so that each trained neural network can process the image to be recognized in real time, thereby improving the efficiency of image processing.

[0170] Figure 8 This is a functional structure diagram of the model training module provided in an embodiment of this application. Figure 8 As shown:

[0171] The model training module 801 can be used to train the object detection model 802 and the text recognition model 803; for example, the object detection model 802 may include the object detection network 4 including the feature extraction module 401 and the feature fusion module 402 in the foregoing embodiments; the text recognition model 803 may include the recognition network in the foregoing embodiments.

[0172] For example, the model training module 801 can train the object detection model 802 and the text recognition model 803 based on various sample data. After the object detection model 802 and the text recognition model 803 are trained, the object detection model 802 can perform object detection and orientation detection on the input image to be recognized. For example, if no target object is detected, the output module 804 can output an invalid result. If the binary segmentation image of the target object and the orientation information of the binary segmentation image are detected, the correction module 805 can perform orientation correction and deformation correction processing on the binary segmentation image to obtain a deformation-corrected image. Then, the text recognition model 803 performs text detection and text recognition on the deformation-corrected image to obtain a text recognition result. After that, the text recognition result is judged. If the judgment result indicates that the target object is compliant, the output module 804 outputs a compliant result. If the judgment result indicates that the target object is non-compliant, the output module 804 outputs an invalid result.

[0173] As can be seen from the above, in this embodiment of the application, the model training module 801 can train different models or neural networks through the same architecture, thereby improving the training efficiency of neural networks.

[0174] Based on the foregoing embodiments, in the recognition method provided in this application, deformation correction is performed on the second image to obtain a deformation-corrected image, which can be used... Figure 9A The process shown is implemented. Figure 9A This is a schematic diagram of the first process for deformation correction of a second image provided in an embodiment of this application. Figure 9A As shown, the process may include steps D1 to D2:

[0175] Step D1: Based on the edge position information of the second image, determine the second contour information of the second graphic whose outer edge position information is determined.

[0176] The second figure includes a closed figure composed of multiple straight lines connected end to end in sequence.

[0177] For example, the method for determining the second contour information can be the same as the method for determining the first contour information in the aforementioned embodiments, and will not be repeated here.

[0178] Step D2: Based on the edge position information and the second contour information, perform deformation correction on the second image to obtain a deformation-corrected image.

[0179] In one implementation, deformation correction is performed on the second image based on edge location information and second contour information to obtain a deformation-corrected image, which can be achieved in the following way:

[0180] Based on the difference between the corresponding pixel coordinates in the edge position information and the second contour information, the second image is subjected to deformation correction by means of pixel stretching, pixel smoothing, or pixel interpolation to obtain a deformation-corrected image.

[0181] As can be seen from the above, in the recognition method provided in this application embodiment, when the target object is a sign, the second image obtained by processing the image to be recognized is deformed to obtain a deformed image based on the characteristic that the sign has a standard geometric shape, and then the recognition result is obtained by recognizing it. This makes the deformation correction processing of the second image more consistent with the actual shape characteristics of the sign, thereby further improving the accuracy of the recognition result.

[0182] Based on the foregoing embodiments, the recognition method provided in this application performs deformation correction on the second image based on edge location information and second contour information, which can be achieved through... Figure 9B The process shown is implemented as follows: Figure 9B This is a schematic diagram of the second process for deform correction of a second image provided in an embodiment of this application, as shown below. Figure 9B As shown, the process may include steps E1 to E2:

[0183] Step E1: Obtain the fitting degree information between the edge position information and the second contour information.

[0184] In one embodiment, the fitting degree information may include the difference in coordinate values ​​between the edge position information and the corresponding pixel in the second contour information; for example, the fitting degree information may also include the mean square error between the edge position information and the corresponding pixel in the second contour information.

[0185] For example, the edge location information can be Figure 3B The closed curve 301 in the middle, the second contour information can be Figure 3B The four sides of the circumscribed rectangle 302 in the diagram. Figure 3B The pixel set 3011 in the closed curve 301 can be a partial contour point set of the curve 301.

[0186] For example, the fit information can be represented by the long side p2p3 of the circumscribed rectangle 302 corresponding to the pixel set 3011, and the mean square error between the pixel set 3011; for example, the long side p2p3 can be the side connecting the third corner point 3023 and the fourth corner point 3024. For example, the long side p2p3 can be represented by equation (1):

[0187] ax + by + c = 0 (1)

[0188] Where a = y3 - y2, b = x2 - x3, c = x3 * y2 - x2 * y3. For example, let Pdown denote the curve containing pixel set 3011. Then, the point p(x...) in Pdown... p y p The distance d between the corresponding points on line segment p2 and p3 p It can be calculated using equation (2):

[0189]

[0190] Then, the point (x) in Pdown p y p The mean square error d between corresponding points in line segment p2p3 and the line segment p3 MSE It can be calculated using equation (3):

[0191]

[0192] Where N is the number of pixels in the pixel set 3011.

[0193] Step E2: If the fitting degree information is greater than the fitting threshold, the second image is deformed by fitting a Bézier curve; if the fitting degree information is less than or equal to the fitting threshold, the second image is deformed by transmission transformation.

[0194] For example, a fitting threshold can be preset. If the fitting degree information is greater than the fitting threshold, the coordinate transformation relationship between the edge position information and the second contour information can be calculated based on curve fitting. That is, Bezier parameters are calculated for the edge position information and the second contour information, and W points are sampled for the edge position information and the second contour information to obtain a set of W pairs of points Pn. Each pair of points corresponds to a column of pixels in the transformed image. Here, W is the width of the transformed image, i.e., the width of the deformation correction image, and the sampling frequency on the set of W pairs of points is the height H of the transformed image, i.e., the height of the deformation correction image. Here, W is an integer greater than 1 and less than or equal to the number of pixels contained in the edge position information; H is an integer greater than 1 and less than or equal to the height of the second contour information.

[0195] For example, if the fitting degree information is less than or equal to the fitting threshold, the rBox function in OpenCV can be used to calculate the transmission transformation matrix based on the edge position information and the corresponding corner points in the second contour information. Then, the second image can be deformed based on the transmission transformation matrix to obtain a deformed image.

[0196] In related technologies, image deformation correction is usually achieved through transmission transformation of the homography matrix. This method cannot correct image deformation caused by non-coplanarity of target objects and camera distortion. However, the recognition method provided in this application adopts different deformation correction methods based on the relationship between fitting degree information and fitting threshold, which can effectively solve the above problems.

[0197] As can be seen from the above, the recognition method provided in this application embodiment can perform targeted deformation correction on the second image based on the fitting degree information between the edge position information of the second image and the second contour information of the second graphic of the external second image, thereby making the characteristics of the target object in the deformation-corrected image clearer and thus improving the recognition accuracy of the target object.

[0198] Based on the foregoing embodiments, this application also provides an identification device 10. Figure 10 This is a schematic diagram of the structure of the identification device 10 provided in an embodiment of this application. Figure 10 The identification device 10 may include:

[0199] The acquisition module 1001 is used to acquire an image to be identified that contains at least the target object.

[0200] The detection module 1002 is used to perform target detection on the image to be recognized to obtain a first image, and to perform orientation detection on the first image to obtain orientation information of the first image; wherein, the first image includes an image containing the target object obtained by segmenting from the image to be recognized;

[0201] The processing module 1003 is used to perform orientation correction processing on the first image based on the orientation information to obtain the second image.

[0202] The recognition module 1004 is used to recognize the second image and obtain the recognition result of the target object.

[0203] In one embodiment, the processing module 1003 is used to determine the first contour information of the first graphic that is connected to the edge of the first image; based on the direction information, the corner position information in the first contour information is subjected to orientation correction processing to obtain the second image; wherein, the first graphic includes a closed graphic composed of multiple straight lines connected end to end in sequence.

[0204] In one embodiment, the detection module 1002 is used to perform target detection on the image to be recognized through the feature extraction module in the trained target detection network to obtain a binary segmentation image; wherein, the feature extraction module includes MobilenetV2; the feature extraction module includes multiple feature extraction units.

[0205] In one embodiment, the detection module 1002 is used to perform feature fusion on the feature extraction results output by at least two feature extraction units of the feature extraction module and the binary segmented image through the feature fusion module in the trained target detection network to obtain the orientation information of the first image.

[0206] In one embodiment, the recognition device 10 further includes a training module for obtaining sample data containing annotation information; wherein the sample data includes image data containing a target object; the annotation information includes the position information of the target object in the image data and the orientation information of the image region containing the target object;

[0207] The training module is also used to perform target detection on the sample data through the feature extraction module to obtain a binary segmentation image corresponding to the sample data; to perform feature fusion on the binary segmentation image corresponding to the sample data and the feature extraction results output by at least two feature extraction units of the feature extraction module through the feature fusion module to obtain the orientation information corresponding to the sample data; and to adjust the parameters of the feature extraction module and the feature fusion module based on the annotation information, the binary segmentation image corresponding to the sample data, and the orientation information corresponding to the sample data to obtain the trained target detection network.

[0208] In one embodiment, the processing module 1003 is used to perform deformation correction on the second image to obtain a deformation-corrected image;

[0209] The recognition module 1004 is used to recognize the deformation-corrected image and obtain the recognition result.

[0210] In one embodiment, the processing module 1003 is used to determine the second contour information of the second graphic with circumscribed edge position information based on the edge position information of the second image; and to perform deformation correction on the second image based on the edge position information and the second contour information to obtain a deformation-corrected image; wherein the second graphic includes a closed graphic composed of multiple straight lines connected end to end in sequence.

[0211] In one embodiment, the processing module 1003 is used to obtain fitting degree information between edge position information and second contour information; if the fitting degree information is greater than the fitting threshold, the second image is deformed by Bézier curve fitting; if the fitting degree information is less than or equal to the fitting threshold, the second image is deformed by transmission transformation.

[0212] Based on the foregoing embodiments, this application also provides an identification device 11. Figure 11 This is a schematic diagram of the structure of the identification device 11 provided in an embodiment of this application. Figure 11 As shown, the identification device 11 may include a processor 1101 and a memory 1102.

[0213] It should be noted that the processor 1101 described above can be at least one of ASIC, DSP, DSPD, PLD, FPGA, CPU, controller, microcontroller, and microprocessor. The memory 1102 described above can be volatile memory, such as Random Access Memory (RAM); or non-volatile memory, such as Read-Only Memory (ROM), flash memory, Hard Disk Drive (HDD), or Solid State Disk (SSD); or a combination of the above types of memory, providing instructions and data to the processor 1101. For example, the acquisition module 1001, detection module 1002, processing module 1003, and identification module 1004 in the foregoing embodiments can be implemented using the processor 1101.

[0214] Based on the foregoing embodiments, this application also provides a computer-readable storage medium storing a computer program that, when executed by a processor of an electronic device, can implement the identification method as described in any of the preceding embodiments.

[0215] The description of the various embodiments above tends to emphasize the differences between the various embodiments. The similarities or similarities between them can be referred to, and for the sake of brevity, they will not be repeated here.

[0216] The methods disclosed in the various method embodiments provided in this application can be arbitrarily combined to obtain new method embodiments without conflict.

[0217] The features disclosed in the various product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0218] The features disclosed in the various method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method or device embodiments.

[0219] It should be noted that the aforementioned computer-readable storage media can be ROM, Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Ferromagnetic Random Access Memory (FRAM), Flash Memory, Magnetic Surface Memory, Optical Disc, or Compact Disc Read-Only Memory (CD-ROM), etc.; or it can be various electronic devices including one or any combination of the above-mentioned memories, such as mobile phones, computers, tablet devices, personal digital assistants, etc.

[0220] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0221] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0222] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware nodes. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0223] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0224] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0225] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0226] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A method of identification, characterized in that, The method includes: Obtain an image to be identified that contains at least the target object; Target detection is performed on the image to be identified to obtain a first image; wherein, the first image includes an image containing the target object obtained by segmenting from the image to be identified; The first image is subjected to orientation detection to obtain the orientation information of the first image; Based on the direction information, the first image is subjected to orientation correction processing to obtain the second image; The second image is then analyzed to obtain the identification result of the target object; The step of performing orientation correction processing on the first image based on the orientation information to obtain the second image includes: Determine the first contour information of a first graphic that circumvents the edge of the first image; wherein, the first graphic includes a closed graphic composed of multiple straight lines connected end to end in sequence; Based on the direction information, a starting corner point is determined from multiple corner points of the first contour information, and each corner point is sorted sequentially. Based on the direction information, the sorted corner points and the edges between adjacent corner points in the first contour information are rotated to obtain the second image. The step of recognizing the second image to obtain the recognition result of the target object includes: Based on the edge position information of the second image, the second contour information of the second graphic circumferential to the edge position information is determined; wherein, the second graphic includes a closed graphic composed of multiple straight lines connected end to end in sequence; Based on the edge position information and the second contour information, the degree of fit between the edge position information and the second contour information is determined. Based on the degree of fit information, deformation correction is performed on the second image to obtain a deformation-corrected image. The deformation-corrected image is identified to obtain the identification result; The deformation correction of the second image based on the fitting degree information includes: If the fitting degree information is greater than the fitting threshold, the second image is deformed by Bézier curve fitting. If the fitting degree information is less than or equal to the fitting threshold, the second image is deformed by transmission transformation.

2. The method of claim 1, wherein, The first image includes a binary segmented image containing the target object obtained by segmenting the image to be identified; the step of performing target detection on the image to be identified to obtain the first image includes: The target detection image is obtained by performing target detection on the image to be identified through the feature extraction module in the trained target detection network; wherein, the feature extraction module includes MobilenetV2; the feature extraction module includes multiple feature extraction units.

3. The method of claim 2, wherein, The step of performing orientation detection on the first image to obtain the orientation information of the first image includes: The feature fusion module in the trained target detection network performs feature fusion on the feature extraction results output by at least two feature extraction units of the feature extraction module and the binary segmented image to obtain the orientation information of the first image.

4. The method of claim 3, wherein, The method further includes: Obtain sample data containing annotation information; wherein, the sample data includes image data containing a target object; the annotation information includes the position information of the target object in the image data and the orientation information of the image region containing the target object; The feature extraction module performs target detection on the sample data to obtain a binary segmentation image corresponding to the sample data. The feature fusion module performs feature fusion on the binary segmented image corresponding to the sample data and the feature extraction results output by at least two feature extraction units of the feature extraction module to obtain the direction information corresponding to the sample data. Based on the annotation information, the binary segmentation image corresponding to the sample data, and the orientation information corresponding to the sample data, the parameters of the feature extraction module and the feature fusion module are adjusted to obtain the trained target detection network.

5. An identification device, characterized in that The device includes: The acquisition module is used to acquire an image to be identified that contains at least the target object; The detection module is used to perform target detection on the image to be identified to obtain a first image; and to perform orientation detection on the first image to obtain orientation information of the first image; wherein, the first image includes an image containing the target object segmented from the image to be identified; The processing module is used to perform orientation correction processing on the first image based on the orientation information to obtain the second image; The recognition module is used to recognize the second image and obtain the recognition result of the target object; Specifically, the processing module is used for: Determine the first contour information of a first graphic that circumvents the edge of the first image; wherein, the first graphic includes a closed graphic composed of multiple straight lines connected end to end in sequence; Based on the direction information, a starting corner point is determined from multiple corner points of the first contour information, and each corner point is sorted sequentially. Based on the direction information, the sorted corner points and the edges between adjacent corner points in the first contour information are rotated to obtain the second image. Specifically, the identification module is used for: Based on the edge position information of the second image, the second contour information of the second graphic circumferential to the edge position information is determined; wherein, the second graphic includes a closed graphic composed of multiple straight lines connected end to end in sequence; Based on the edge position information and the second contour information, the degree of fit between the edge position information and the second contour information is determined. Based on the degree of fit information, deformation correction is performed on the second image to obtain a deformation-corrected image. The deformation-corrected image is identified to obtain the identification result; The deformation correction of the second image based on the fitting degree information includes: If the fitting degree information is greater than the fitting threshold, the second image is deformed by Bézier curve fitting. If the fitting degree information is less than or equal to the fitting threshold, the second image is deformed by transmission transformation.

6. An identification device, characterized by The device comprises a processor and a memory; wherein the memory stores a computer program, and the computer program is executed by the processor to implement the identification method according to any one of claims 1 to 4.

7. A computer readable storage medium characterized in that, The computer readable storage medium stores a computer program, and the computer program is executed by the processor of the electronic device to implement the identification method according to any one of claims 1 to 4.

Images