An image correction method, device, computer equipment and storage medium

Abstract

The disclosure provides an image correction method and device, a computer device and a storage device, and the method comprises: obtaining a to-be-corrected image; inputting the to-be-corrected image into a pre-trained neural network to obtain a correction parameter corresponding to the to-be-corrected image; wherein the neural network is trained based on a plurality of sample images, the plurality of sample images comprise a tilted image obtained by tilting a front image, the supervision data of the tilted image is a correction parameter in the tilting process, and the correction parameter comprises a three-dimensional attitude angle; and performing a correction process on the to-be-corrected image based on the correction parameter.

Description

Technical Field

[0001] This disclosure relates to the field of image processing technology, and more specifically, to an image correction method, apparatus, computer device, and storage medium. Background Technology

[0002] Currently, various instruments are widely used in industry and daily life. However, as the number of instruments increases, the workload of reading them also increases exponentially. When reading instruments, images are usually taken manually, and the measurement values ​​are then read from the images. However, since manually taken images may have angular deviations, which can affect the reading results, it is necessary to correct the images.

[0003] In related technologies, when correcting captured images, it is generally necessary to first determine the instrument model, then match the captured image with the standard image of the corresponding model to determine the perspective transformation matrix, and then correct the captured image based on the perspective transformation matrix. This method requires prior knowledge of the instrument model, and cannot be used to correct images for which the instrument model cannot be known, resulting in poor versatility. Summary of the Invention

[0004] This disclosure provides at least one image correction method, apparatus, computer device, and storage medium.

[0005] In a first aspect, embodiments of this disclosure provide an image correction method, including:

[0006] Obtain the image to be corrected;

[0007] The image to be corrected is input into a pre-trained neural network to obtain the correction parameters corresponding to the image to be corrected; wherein, the neural network is trained based on multiple sample images, including a tilted image obtained by tilting a frontal image, and the supervision data of the tilted image is the correction parameters in the tilting process, the correction parameters including three-dimensional pose angles;

[0008] The image to be corrected is processed based on the correction parameters.

[0009] In the methods described above, the correction parameters of the image to be corrected can be determined directly based on a pre-trained neural network, and the image can be corrected based on these parameters. This method has high correction efficiency and accuracy, and is highly versatile. On the other hand, the pre-trained neural network is trained on tilted images, which are obtained by tilting frontal images. Therefore, the training samples for the neural network are relatively abundant, resulting in higher network accuracy.

[0010] In one possible implementation, the method further includes tilting the frontal image to obtain the tilted image according to the following method:

[0011] The target's three-dimensional attitude angles are determined based on a preset range of three-dimensional attitude angles.

[0012] Determine the coordinates of the first vertex of the object to be identified in the frontal image;

[0013] Based on the target's three-dimensional attitude angle, the coordinates of the first vertex are transformed to generate the coordinates of the second vertex;

[0014] A first perspective transformation matrix is ​​determined based on the first vertex coordinates and the second vertex coordinates, and the frontal image is processed based on the first perspective transformation matrix to obtain the tilted image.

[0015] This method determines multiple target three-dimensional attitude angles within a preset range of three-dimensional attitude angles. Based on any frontal image, multiple tilted images can be generated, enriching the sample data for neural network training. As a result, the neural network trained in this way has higher network accuracy.

[0016] In one possible implementation, the step of transforming the coordinates of the first vertex based on the target three-dimensional attitude angle to generate the coordinates of the second vertex includes:

[0017] Based on the target's three-dimensional attitude angle, the coordinates of the first vertex are transformed to determine the target vertex coordinates;

[0018] The target distance is determined based on a preset distance range between the image acquisition device and the object to be identified.

[0019] The position information of the imaging plane is determined based on the target distance, and the projection point of the intermediate vertex coordinates on the imaging plane is determined based on the intermediate vertex coordinates and the focal position of the image acquisition device. The position coordinates of the projection point are the second vertex coordinates.

[0020] By combining different target distances when generating tilted images, tilted images at different shooting distances and angles can be generated, enriching the variety of sample data used for neural network training.

[0021] In one possible implementation, after generating the coordinates of the second vertex, the method further includes:

[0022] Determine whether the second vertex is within the preset vertex coordinate range;

[0023] If so, a first perspective transformation matrix is ​​determined based on the first vertex coordinates and the second vertex coordinates, and the frontal image is processed based on the first perspective transformation matrix.

[0024] This implementation method allows the object to be identified to be fully presented in the tilted image after tilting, thus improving the quality of the sample data.

[0025] In one possible implementation, the method further includes:

[0026] After obtaining the tilted image, the target three-dimensional attitude angle is normalized, and the normalized target three-dimensional attitude angle is used as the supervision data of the tilted image.

[0027] This method ensures that the supervisory data of the tilted image and the output data of the neural network maintain a consistent numerical range, facilitating the calculation of the loss value.

[0028] In one possible implementation, the correction processing of the image to be corrected based on the correction parameters includes:

[0029] Based on the correction parameters, determine the second perspective transformation matrix corresponding to the image to be corrected;

[0030] The image to be corrected is processed based on the second perspective transformation matrix to obtain the corrected target image.

[0031] In one possible implementation, determining the second perspective transformation matrix corresponding to the image to be corrected based on the correction parameters includes:

[0032] Determine the coordinates of the third vertex of the object to be identified in the image to be corrected;

[0033] Based on the three-dimensional attitude angle in the correction parameters, the coordinates of the third vertex are transformed to obtain the coordinates of the fourth vertex;

[0034] The second perspective transformation matrix is ​​determined based on the coordinates of the third vertex and the coordinates of the fourth vertex.

[0035] This method allows for the automatic calculation of the second perspective transformation matrix, improving the efficiency of image correction.

[0036] In one possible implementation, the image to be corrected is an image containing an instrument;

[0037] The method further includes:

[0038] The corrected target image corresponding to the image to be corrected is input into a pre-trained reading recognition network to determine the measurement value of the instrument in the image to be corrected.

[0039] This method can automatically output instrument measurement values, improving the efficiency and accuracy of instrument measurement readings.

[0040] Secondly, embodiments of this disclosure also provide an image correction device, comprising:

[0041] The acquisition module is used to acquire the image to be corrected.

[0042] A prediction module is used to input the image to be corrected into a pre-trained neural network to obtain the correction parameters corresponding to the image to be corrected; wherein, the neural network is trained based on multiple sample images, the multiple sample images include a tilted image obtained by tilting a frontal image, the supervision data of the tilted image is the correction parameters in the tilting process, and the correction parameters include three-dimensional pose angles;

[0043] The correction module is used to perform correction processing on the image to be corrected based on the correction parameters.

[0044] In one possible implementation, the apparatus further includes a training module for tilting the frontal image to obtain the tilted image according to the following method:

[0045] The target's three-dimensional attitude angles are determined based on a preset range of three-dimensional attitude angles.

[0046] Determine the coordinates of the first vertex of the object to be identified in the frontal image;

[0047] Based on the target's three-dimensional attitude angle, the coordinates of the first vertex are transformed to generate the coordinates of the second vertex;

[0048] A first perspective transformation matrix is ​​determined based on the first vertex coordinates and the second vertex coordinates, and the frontal image is processed based on the first perspective transformation matrix to obtain the tilted image.

[0049] In one possible implementation, the training module, when performing coordinate transformation on the first vertex coordinates based on the target three-dimensional pose angle to generate the second vertex coordinates, is further configured to:

[0050] Based on the target's three-dimensional attitude angle, the coordinates of the first vertex are transformed to determine the target vertex coordinates;

[0051] The target distance is determined based on a preset distance range between the image acquisition device and the object to be identified.

[0052] The position information of the imaging plane is determined based on the target distance, and the projection point of the intermediate vertex coordinates on the imaging plane is determined based on the intermediate vertex coordinates and the focal position of the image acquisition device. The position coordinates of the projection point are the second vertex coordinates.

[0053] In one possible implementation, after generating the coordinates of the second vertex, the training module is further configured to:

[0054] Determine whether the second vertex is within the preset vertex coordinate range;

[0055] If so, a first perspective transformation matrix is ​​determined based on the first vertex coordinates and the second vertex coordinates, and the frontal image is processed based on the first perspective transformation matrix.

[0056] In one possible implementation, the training module is further configured to:

[0057] After obtaining the tilted image, the target three-dimensional attitude angle is normalized, and the normalized target three-dimensional attitude angle is used as the supervision data of the tilted image.

[0058] In one possible implementation, the correction module, when performing correction processing on the image to be corrected based on the correction parameters, is used to:

[0059] Based on the correction parameters, determine the second perspective transformation matrix corresponding to the image to be corrected;

[0060] The image to be corrected is processed based on the second perspective transformation matrix to obtain the corrected target image.

[0061] In one possible implementation, the correction module, when determining the second perspective transformation matrix corresponding to the image to be corrected based on the correction parameters, is used to:

[0062] Determine the coordinates of the third vertex of the object to be identified in the image to be corrected;

[0063] Based on the three-dimensional attitude angle in the correction parameters, the coordinates of the third vertex are transformed to obtain the coordinates of the fourth vertex;

[0064] The second perspective transformation matrix is ​​determined based on the coordinates of the third vertex and the coordinates of the fourth vertex.

[0065] In one possible implementation, the image to be corrected is an image containing an instrument;

[0066] The device further includes an identification module for:

[0067] The corrected target image corresponding to the image to be corrected is input into a pre-trained reading recognition network to determine the measurement value of the instrument in the image to be corrected.

[0068] Thirdly, embodiments of this disclosure also provide a computer device, including: a processor, a memory, and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the computer device is running, the processor communicates with the memory via the bus, and when the machine-readable instructions are executed by the processor, the steps of the first aspect above, or any possible implementation of the first aspect, are performed.

[0069] Fourthly, embodiments of this disclosure also provide a computer-readable storage medium storing a computer program that, when executed by a processor, performs the steps of the first aspect or any possible implementation of the first aspect.

[0070] For a description of the effects of the above-mentioned image correction method, device, computer equipment and storage medium, please refer to the description of the above-mentioned image correction method, which will not be repeated here.

[0071] To make the above-mentioned objects, features and advantages of this disclosure more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0072] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the accompanying drawings used in the embodiments will be briefly described below. These drawings are incorporated in and constitute a part of this specification. They illustrate embodiments conforming to this disclosure and, together with the specification, serve to explain the technical solutions of this disclosure. It should be understood that the following drawings only show some embodiments of this disclosure and should not be considered as limiting the scope. Those skilled in the art can obtain other related drawings based on these drawings without creative effort.

[0073] Figure 1 A flowchart of an image correction method provided by an embodiment of this disclosure is shown;

[0074] Figure 2 This diagram illustrates the relationship between the image coordinate system and the global coordinate system provided in an embodiment of this disclosure.

[0075] Figure 3 A schematic flowchart of a tilt processing method provided in an embodiment of this disclosure is shown;

[0076] Figure 4 An example diagram illustrating the determination of the second vertex coordinates provided in an embodiment of this disclosure is shown;

[0077] Figure 5 A flowchart illustrating the training process of the neural network provided in the embodiments of this disclosure is shown;

[0078] Figure 6 A schematic diagram of an image correction apparatus provided in an embodiment of this disclosure is shown;

[0079] Figure 7 A schematic diagram of the structure of a computer device provided in an embodiment of this disclosure is shown. Detailed Implementation

[0080] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this disclosure, and not all of them. The components of the embodiments of this disclosure described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this disclosure provided in the accompanying drawings is not intended to limit the scope of the claimed disclosure, but merely represents selected embodiments of this disclosure. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without inventive effort are within the scope of protection of this disclosure.

[0081] Research has found that in related technologies, when correcting captured images, it is generally necessary to first determine the instrument model, then match the captured image with the standard image of the corresponding model to determine the perspective transformation matrix, and then correct the captured image based on the perspective transformation matrix. This method requires prior knowledge of the instrument model and cannot be used to correct images for which the instrument model cannot be known, resulting in poor versatility.

[0082] Based on the above research, this disclosure provides an image correction method, apparatus, computer device, and storage medium. It can directly determine the correction parameters of the image to be corrected based on a pre-trained neural network, and then correct the image based on these parameters. This method offers high correction efficiency and accuracy, and strong versatility. Furthermore, the pre-trained neural network is trained on tilted images, which are obtained by tilting frontal images. Therefore, the training samples for the neural network are abundant, resulting in high network accuracy.

[0083] The shortcomings of the above solutions are the result of the inventor's practical experience and careful research. Therefore, the discovery process of the above problems and the solutions proposed in this disclosure below should be considered as the inventor's contribution to this disclosure.

[0084] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0085] To facilitate understanding of this embodiment, a detailed description of the image correction method disclosed in this disclosure is provided first. The execution entity of the image correction method provided in this disclosure is generally a computer device with certain computing capabilities. This computer device may include, for example, a terminal device, a server, or other processing devices. The terminal device may be a user equipment (UE), mobile device, user terminal, terminal, cellular phone, cordless phone, personal digital assistant (PDA), handheld device, computing device, in-vehicle device, wearable device, etc. In some possible implementations, the image correction method can be implemented by a processor calling computer-readable instructions stored in memory.

[0086] See Figure 1 The diagram shows a flowchart of an image correction method provided in an embodiment of this disclosure. The method includes steps 101 to 103, wherein:

[0087] Step 101: Obtain the image to be corrected.

[0088] Step 102: Input the image to be corrected into a pre-trained neural network to obtain the correction parameters corresponding to the image to be corrected; wherein, the neural network is trained based on multiple sample images, the multiple sample images include a tilted image obtained by tilting a frontal image, the supervision data of the tilted image is the correction parameters in the tilting process, and the correction parameters include three-dimensional pose angles.

[0089] Step 103: Perform correction processing on the image to be corrected based on the correction parameters.

[0090] The following is a detailed explanation of the steps described above:

[0091] Regarding step 101,

[0092] In one possible implementation, acquiring the image to be corrected may refer to acquiring an image taken by the user, which may contain an object to be identified, such as an instrument, table, or chair.

[0093] In one possible implementation, when the image correction method provided in this disclosure is applied to a server, obtaining the image to be corrected may refer to receiving the image to be corrected uploaded by the user terminal, and after the image to be corrected is processed based on the image correction method provided in this disclosure, the server sends the corrected target image to the user terminal.

[0094] Regarding step 102,

[0095] The correction parameters are the parameters used when correcting the image to be corrected. For example, the correction parameters may include three-dimensional pose angle, distance between the image acquisition device and the object to be identified (i.e., the shooting distance corresponding to the object to be identified), etc.

[0096] The three-dimensional attitude angles may include, for example, roll angle, yaw angle, and pitch angle in an image coordinate system. The image coordinate system can be a three-dimensional coordinate system based on the image to be corrected, in which the Z-axis of each pixel in the image to be corrected is labeled as 0; or, the three-dimensional attitude angles may include roll angle, yaw angle, and pitch angle in a global coordinate system, where the X and Y axes of the global coordinate system coincide with the image coordinate system, and the image coordinate system is a two-dimensional coordinate system based on the image to be corrected.

[0097] The range of the three-dimensional attitude angles can be, for example, [-15°, 15°].

[0098] The neural network is trained using multiple sample images, including tilted images obtained by tilting a frontal image. The supervision data for the tilted images are the correction parameters in the tilting process. Generally, the supervision data for the tilted images are the target three-dimensional pose angles in the tilting process.

[0099] Specifically, the training process of the neural network will be described in detail below, and will not be explained in detail here.

[0100] Regarding step 103,

[0101] In one possible implementation, when performing correction processing on the image to be corrected based on the correction parameters, the second perspective transformation matrix corresponding to the image to be corrected can be determined first based on the correction parameters; then, the image to be corrected can be performed based on the second perspective transformation matrix to obtain the corrected target image.

[0102] Specifically, when determining the second perspective transformation matrix corresponding to the image to be corrected based on the correction parameters, the coordinates of the third vertex of the object to be identified in the image to be corrected can be determined first. Then, based on the three-dimensional pose angle in the correction parameters, the coordinates of the third vertex are transformed to obtain the coordinates of the fourth vertex. Finally, the second perspective transformation matrix is ​​determined based on the coordinates of the third vertex and the coordinates of the fourth vertex.

[0103] The third vertex coordinates can refer to the coordinates in the image coordinate system corresponding to the image to be corrected. The image coordinate system can be, for example, a coordinate system with the upper left corner as the origin. In one possible implementation, when determining the third vertex coordinates of the object to be identified in the image to be corrected, key point recognition can be performed on the image to be corrected to determine the vertex of the object to be identified and the third vertex coordinates of the vertex.

[0104] Taking the instrument as an example, the coordinates of the third vertex of the object to be identified are the coordinates corresponding to the four vertices of the instrument.

[0105] Here, it should be noted that the coordinates of the third vertex are the coordinates of the object to be identified in the image coordinate system corresponding to the image to be corrected, and the coordinates of the fourth vertex are the coordinates of the object to be identified in the global coordinate system. The X-axis and Y-axis of the global coordinate system and the image coordinate system can coincide. The image coordinate system can be a two-dimensional coordinate system or a three-dimensional coordinate system. For example, the relationship between the image coordinate system and the global coordinate system can be as follows: Figure 2 As shown.

[0106] The global coordinate system is defined by O as the origin and the X, Y, and Z axes as the coordinate axes; the image coordinate system is defined by O as the origin and the X', Y', and Z' axes as the coordinate axes; the black solid points P1, P2, P3, and P4 represent the coordinates of the third vertex; and the black solid points P1', P2', P3', and P4' represent the coordinates of the fourth vertex.

[0107] In one possible implementation, when performing coordinate transformation on the coordinates of the third vertex based on the three-dimensional attitude angle in the correction parameters, the coordinates of the third vertex can first be rotated around the X-axis, Y-axis and Z-axis respectively based on the three-dimensional attitude angle in the correction parameters. The rotation angle is the value of the three-dimensional attitude angle in the correction parameters. After the rotation, the values ​​of each vertex in the global coordinate system are the coordinates of the fourth vertex.

[0108] In one possible implementation, when determining the second perspective transformation matrix based on the coordinates of the third vertex and the fourth vertex, the coordinates of the third vertex and the fourth vertex can be input into a tool library such as OpenCV to determine the second perspective transformation matrix.

[0109] It should be noted that the second perspective transformation matrix means that after the frontal image is tilted by the second perspective transformation matrix, the image to be corrected is obtained. Therefore, when the image to be corrected is corrected based on the second perspective transformation matrix to obtain the corrected target image, the inverse matrix of the second perspective transformation matrix can be calculated first, and then the image to be corrected can be corrected based on the inverse matrix to obtain the corrected target image.

[0110] When the object to be identified in the image to be corrected is an instrument, after the image to be corrected is corrected, the corrected target image can be displayed on the user terminal, and the user can manually read the measurement value of the instrument in the target image; or, in another possible implementation, the target image can be input into a pre-trained reading recognition network to determine the measurement value of the instrument in the image to be corrected.

[0111] For example, during training, the reading recognition network can acquire multiple first sample images containing the instrument. The first sample images carry data tags, which are used to represent the measurement values ​​within the instrument. Then, the first sample images are input into the reading recognition network to be trained to determine the predicted measurement values ​​of the reading recognition network for the first sample images. Based on the data tags and predicted measurement values ​​of the first sample images, the loss value for this training is determined, and the reading recognition network to be recognized is trained based on the loss value.

[0112] This method can automatically output instrument measurement values, improving the efficiency and accuracy of instrument measurement readings.

[0113] The training process of the neural network will be described below, taking the instrument as an example.

[0114] Before training the neural network, sample images for training the neural network can be acquired first.

[0115] In practical applications, considering that the number of sample images may be small, in order to avoid the network accuracy of the neural network being affected by the number of sample images, data augmentation processing can be performed on the initially acquired sample images.

[0116] For example, multiple front images of the instrument can be acquired first, and then the front images can be tilted.

[0117] Reference Figure 3 The diagram shown is a flowchart of a tilt processing method provided in this disclosure, which includes the following steps:

[0118] Step 301: Determine the target three-dimensional attitude angle based on the preset three-dimensional attitude angle range.

[0119] Specifically, when determining the target three-dimensional attitude angle, three coefficients can be randomly generated from the preset three-dimensional attitude angle range as the target three-dimensional attitude angle.

[0120] Step 302: Determine the coordinates of the first vertex of the object to be identified in the frontal image.

[0121] Specifically, the method for determining the coordinates of the first vertex of the object to be identified in the frontal image can be the same as the method for determining the coordinates of the third vertex of the object to be identified in the image to be corrected, and will not be described again here.

[0122] Step 303: Based on the target three-dimensional attitude angle, perform coordinate transformation on the coordinates of the first vertex to generate the coordinates of the second vertex.

[0123] This method determines multiple target three-dimensional attitude angles within a preset range of three-dimensional attitude angles. Based on any frontal image, multiple tilted images can be generated, enriching the sample data for neural network training. As a result, the neural network trained in this way has higher network accuracy.

[0124] Specifically, when generating the second vertex coordinates by transforming the first vertex coordinates based on the target three-dimensional attitude angle, the target vertex coordinates can be determined by transforming the first vertex coordinates based on the three-dimensional attitude angle; then, the target distance is determined based on the preset distance range between the image acquisition device and the object to be identified; finally, the first position information of the imaging plane is determined based on the target distance, and the first projection point of the target vertex coordinates on the imaging plane is determined based on the intermediate vertex coordinates and the focal position of the image acquisition device, and the position coordinates of the first projection point are the second vertex coordinates.

[0125] By combining different target distances when generating tilted images, tilted images at different shooting distances and angles can be generated, enriching the variety of sample data used for neural network training.

[0126] Specifically, when determining the target vertex coordinates, the first vertex coordinates can be controlled to rotate around the X-axis, Y-axis and Z-axis respectively, and the rotation angle is the value of the target three-dimensional attitude angle. After rotation, the value of each vertex in the global coordinate system is the target vertex coordinate.

[0127] When determining the target distance based on the preset distance range between the image acquisition device and the object to be identified, a value can be randomly determined within the preset distance range and this value can be determined as the target distance.

[0128] When determining the position information of the imaging plane based on the target distance, a plane that is at the target distance from the frontal image can be used as the imaging plane. The focal position of the image acquisition device is a position at a distance from the imaging plane equal to the focal length.

[0129] For example, when determining the coordinates of the second vertex, it can be done as follows: Figure 4 As shown, the distance between the imaging plane and the focal point is the focal length f, the coordinates of the black solid point are the coordinates of the target vertex, and the coordinates of the gray solid point are the coordinates of the second vertex.

[0130] In one possible implementation, since the size of the imaging plane is limited, after generating the second vertex coordinates, in order to avoid the transformed vertex not falling on the imaging plane, before determining the first perspective transformation matrix based on the first vertex coordinates and the second vertex coordinates, it can also be determined whether the second vertex coordinates are within a preset vertex coordinate range. If the second vertex coordinates are within the preset vertex coordinate range, then step 304 is executed.

[0131] In another possible implementation, if the coordinates of the second vertex are not within the preset vertex coordinate range, the process can return to the step of determining the target three-dimensional attitude angle and redetermine the target three-dimensional attitude angle.

[0132] Specifically, when determining whether the coordinates of the second vertex are within the preset range of vertex coordinates, a minimum bounding rectangle can be drawn based on the coordinates of the second vertex; then it is determined whether the bounding rectangle appears completely within the imaging plane; if the bounding rectangle appears completely within the imaging plane, the subsequent steps are executed; if the bounding rectangle does not appear completely within the imaging plane, the process returns to the step of determining the target's three-dimensional attitude angle.

[0133] This implementation method allows the object to be identified to be fully presented in the tilted image after tilting, thus improving the quality of the sample data.

[0134] Step 304: Determine the first perspective transformation matrix based on the first vertex coordinates and the second vertex coordinates, and process the frontal image based on the first perspective transformation matrix to obtain the tilted image.

[0135] In one possible implementation, when determining the first perspective transformation matrix based on the first vertex coordinates and the second vertex coordinates, the first vertex coordinates and the second vertex coordinates can be input into a tool library such as OpenCV to determine the first perspective transformation matrix.

[0136] Furthermore, since the output value of the neural network is in the range of 0 to 1, in order to ensure the consistency between the supervision data and the output value of the neural network, the supervision data can be normalized. That is, the target three-dimensional attitude angle applied in the tilting process is normalized, and the normalized target three-dimensional attitude angle is used as the supervision data of the tilted image.

[0137] This method ensures that the supervisory data of the tilted image and the output data of the neural network maintain a consistent numerical range, facilitating the calculation of the loss value.

[0138] After tilting the frontal image, the size of the object to be identified often changes. Therefore, in order to reduce the impact of the background area of ​​the tilted image on the recognition accuracy of the neural network, the background color of the tilted image can also be filled.

[0139] In one possible implementation, the background color to be filled can be a preset color, or it can be the background color of the front image.

[0140] In one possible implementation, the training process of the neural network can be as follows: Figure 5 As shown, it includes the following steps:

[0141] Step 501: Obtain the tilted image and the target's three-dimensional attitude angle corresponding to the tilted image.

[0142] Wherein, the target three-dimensional attitude angle is the three-dimensional attitude angle applied during the tilting process of the tilted image.

[0143] Step 502: Input the tilted image into the neural network to be trained to obtain the predicted three-dimensional pose angle corresponding to the tilted image.

[0144] Step 503: Determine the loss value in this training process based on the predicted three-dimensional attitude angle and the target three-dimensional attitude angle, and train the neural network to be trained based on the loss value.

[0145] When determining the loss value during the training process based on the predicted three-dimensional attitude angle and the target three-dimensional attitude angle, the cross-entropy loss between the two can be calculated as an example.

[0146] In the aforementioned image correction methods, the correction parameters of the image to be corrected can be determined directly based on a pre-trained neural network, and the image can be corrected based on these parameters. This method has high correction efficiency and accuracy, and strong versatility. On the other hand, the pre-trained neural network is trained on tilted images, which are obtained by tilting frontal images. Therefore, the training samples for the neural network are relatively abundant, resulting in high network accuracy.

[0147] Those skilled in the art will understand that, in the above-described method of the specific implementation, the order in which each step is written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0148] Based on the same inventive concept, this disclosure also provides an image correction device corresponding to the image correction method. Since the principle of the device in this disclosure for solving the problem is similar to that of the image correction method described above in this disclosure, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be described again.

[0149] Reference Figure 6 The diagram illustrates the architecture of an image correction device according to an embodiment of this disclosure. The device includes: an acquisition module 601, a prediction module 602, a correction module 603, a training module 604, and a recognition module 605, wherein:

[0150] Acquisition module 601 is used to acquire the image to be corrected;

[0151] The prediction module 602 is used to input the image to be corrected into a pre-trained neural network to obtain the correction parameters corresponding to the image to be corrected; wherein, the neural network is trained based on multiple sample images, the multiple sample images include a tilted image obtained by tilting a frontal image, the supervision data of the tilted image is the correction parameters in the tilting process, and the correction parameters include three-dimensional pose angles.

[0152] The correction module 603 is used to perform correction processing on the image to be corrected based on the correction parameters.

[0153] In one possible implementation, the apparatus further includes a training module 604 for tilting the frontal image to obtain the tilted image according to the following method:

[0154] The target's three-dimensional attitude angles are determined based on a preset range of three-dimensional attitude angles.

[0155] Determine the coordinates of the first vertex of the object to be identified in the frontal image;

[0156] Based on the target's three-dimensional attitude angle, the coordinates of the first vertex are transformed to generate the coordinates of the second vertex;

[0157] A first perspective transformation matrix is ​​determined based on the first vertex coordinates and the second vertex coordinates, and the frontal image is processed based on the first perspective transformation matrix to obtain the tilted image.

[0158] In one possible implementation, the training module 604, when performing coordinate transformation on the first vertex coordinates based on the target three-dimensional pose angle to generate the second vertex coordinates, is further configured to:

[0159] Based on the target's three-dimensional attitude angle, the coordinates of the first vertex are transformed to determine the target vertex coordinates;

[0160] The target distance is determined based on a preset distance range between the image acquisition device and the object to be identified.

[0161] The position information of the imaging plane is determined based on the target distance, and the projection point of the intermediate vertex coordinates on the imaging plane is determined based on the intermediate vertex coordinates and the focal position of the image acquisition device. The position coordinates of the projection point are the second vertex coordinates.

[0162] In one possible implementation, after generating the coordinates of the second vertex, the training module 604 is further configured to:

[0163] Determine whether the second vertex is within the preset vertex coordinate range;

[0164] If so, a first perspective transformation matrix is ​​determined based on the first vertex coordinates and the second vertex coordinates, and the frontal image is processed based on the first perspective transformation matrix.

[0165] In one possible implementation, the training module 604 is further configured to:

[0166] After obtaining the tilted image, the target three-dimensional attitude angle is normalized, and the normalized target three-dimensional attitude angle is used as the supervision data of the tilted image.

[0167] In one possible implementation, the correction module 603, when performing correction processing on the image to be corrected based on the correction parameters, is used to:

[0168] Based on the correction parameters, determine the second perspective transformation matrix corresponding to the image to be corrected;

[0169] The image to be corrected is processed based on the second perspective transformation matrix to obtain the corrected target image.

[0170] In one possible implementation, the correction module 603, when determining the second perspective transformation matrix corresponding to the image to be corrected based on the correction parameters, is used to:

[0171] Determine the coordinates of the third vertex of the object to be identified in the image to be corrected;

[0172] Based on the three-dimensional attitude angle in the correction parameters, the coordinates of the third vertex are transformed to obtain the coordinates of the fourth vertex;

[0173] The second perspective transformation matrix is ​​determined based on the coordinates of the third vertex and the coordinates of the fourth vertex.

[0174] In one possible implementation, the image to be corrected is an image containing an instrument;

[0175] The device further includes an identification module 605, used for:

[0176] The corrected target image corresponding to the image to be corrected is input into a pre-trained reading recognition network to determine the measurement value of the instrument in the image to be corrected.

[0177] The processing flow of each module in the device and the interaction flow between each module can be referred to the relevant descriptions in the above method embodiments, and will not be detailed here.

[0178] Based on the same technical concept, this disclosure also provides a computer device. (See also...) Figure 7 The diagram shows the structure of a computer device 700 provided in this embodiment, including a processor 701, a memory 702, and a bus 703. The memory 702 stores execution instructions and includes main memory 7021 and external memory 7022. The main memory 7021, also called internal memory, is used to temporarily store computational data in the processor 701 and data exchanged with external memory 7022 such as a hard disk. The processor 701 exchanges data with the external memory 7022 through the main memory 7021. When the computer device 700 is running, the processor 701 and the memory 702 communicate through the bus 703, causing the processor 701 to execute the following instructions:

[0179] Obtain the image to be corrected;

[0180] The image to be corrected is input into a pre-trained neural network to obtain the correction parameters corresponding to the image to be corrected; wherein, the neural network is trained based on multiple sample images, including a tilted image obtained by tilting a frontal image, and the supervision data of the tilted image is the correction parameters in the tilting process, the correction parameters including three-dimensional pose angles;

[0181] The image to be corrected is processed based on the correction parameters.

[0182] This disclosure also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of the image correction method described in the above-described method embodiments. The storage medium may be a volatile or non-volatile computer-readable storage medium.

[0183] This disclosure also provides a computer program product carrying program code. The program code includes instructions that can be used to execute the steps of the image correction method described in the above method embodiments. For details, please refer to the above method embodiments, which will not be repeated here.

[0184] The aforementioned computer program product can be implemented through hardware, software, or a combination thereof. In one optional embodiment, the computer program product is specifically embodied in a computer storage medium; in another optional embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDK), etc.

[0185] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems and devices described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. In the several embodiments provided in this disclosure, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division; in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Another point is that the displayed or discussed mutual coupling or direct coupling or communication connection may be through some communication interfaces; the indirect coupling or communication connection of devices or units may be electrical, mechanical, or other forms.

[0186] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0187] In addition, the functional units in the various embodiments of this disclosure can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0188] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of this disclosure, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this disclosure. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0189] Finally, it should be noted that the above-described embodiments are merely specific implementations of this disclosure, used to illustrate the technical solutions of this disclosure, and not to limit it. The protection scope of this disclosure is not limited thereto. Although this disclosure has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features, within the scope of the technology disclosed in this disclosure. Such modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this disclosure, and should all be covered within the protection scope of this disclosure. Therefore, the protection scope of this disclosure should be determined by the protection scope of the claims.

Claims

1. An image correction method characterized by, The method comprises: obtaining a to-be-corrected image; the to-be-corrected image is an image containing an instrument; inputting the to-be-corrected image into a pre-trained neural network to obtain a correction parameter corresponding to the to-be-corrected image; wherein the neural network is trained based on a plurality of sample images, the plurality of sample images include a tilted image obtained by tilting a front image, the supervision data of the tilted image is a correction parameter in the tilting process, and the correction parameter includes a three-dimensional attitude angle and a distance between an image acquisition device and an instrument in the front image; based on the correction parameter, correcting the to-be-corrected image; the method further comprises tilting the front image to obtain the tilted image according to the following method: determining a target three-dimensional attitude angle based on a preset three-dimensional attitude angle range; determining a first vertex coordinate of a to-be-recognized object in the front image; based on the target three-dimensional attitude angle, performing coordinate transformation on the first vertex coordinate to generate a second vertex coordinate; determining a first perspective transformation matrix based on the first vertex coordinate and the second vertex coordinate, and processing the front image based on the first perspective transformation matrix to obtain the tilted image; the method further comprises: based on the target three-dimensional attitude angle, performing coordinate transformation on the first vertex coordinate to determine a target vertex coordinate; determining a target distance based on a preset distance range between an image acquisition device and the to-be-recognized object; determining the position information of the imaging plane based on the target distance, and determining the projection point of the target vertex coordinate on the imaging plane based on the target vertex coordinate and the focal point position of the image acquisition device, the position coordinate of the projection point being the second vertex coordinate.

2. The method of claim 1, wherein, After generating the second vertex coordinate, the method further comprises: determining whether the second vertex is within a preset vertex coordinate range; if yes, determining a first perspective transformation matrix based on the first vertex coordinate and the second vertex coordinate, and processing the front image based on the first perspective transformation matrix.

3. The method according to any one of claims 1 to 2, characterized in that, the method further comprises: after obtaining the tilted image, normalizing the target three-dimensional attitude angle, and using the normalized target three-dimensional attitude angle as the supervision data of the tilted image.

4. The method of claim 3, wherein, the method further comprises: based on the correction parameter, determining a second perspective transformation matrix corresponding to the to-be-corrected image; based on the second perspective transformation matrix, correcting the to-be-corrected image to obtain a target image after correction processing.

5. The method of claim 4, wherein, the method further comprises: determining a third vertex coordinate of a to-be-recognized object in the to-be-corrected image; based on the three-dimensional attitude angle in the correction parameter, performing coordinate transformation on the third vertex coordinate to obtain a fourth vertex coordinate; determining the second perspective transformation matrix based on the third vertex coordinate and the fourth vertex coordinate.

6. The method of claim 5, wherein, the method further comprises: inputting the target image after the correction processing corresponding to the image to be corrected into a pre-trained reading identification network to determine the measurement value of the instrument in the image to be corrected.

7. An image correction apparatus characterized by comprising: comprising: an acquisition module, configured to acquire an image to be corrected; the image to be corrected is an image containing an instrument; a prediction module, configured to input the image to be corrected into a pre-trained neural network to obtain a correction parameter corresponding to the image to be corrected; wherein the neural network is trained based on a plurality of sample images, the plurality of sample images include a tilted image obtained by tilting a front image, the supervision data of the tilted image is a correction parameter in the tilting process, the correction parameter includes a three-dimensional attitude angle and a distance between an image acquisition device and an instrument in the front image; a correction module, configured to perform correction processing on the image to be corrected based on the correction parameter; the device further comprises a method for tilting the front image to obtain the tilted image: determine a target three-dimensional attitude angle based on a preset three-dimensional attitude angle range; determine a first vertex coordinate of an object to be identified in the front image; perform coordinate transformation on the first vertex coordinate based on the target three-dimensional attitude angle to generate a second vertex coordinate; determine a first perspective transformation matrix based on the first vertex coordinate and the second vertex coordinate, and process the front image based on the first perspective transformation matrix to obtain the tilted image; the coordinate transformation on the first vertex coordinate based on the target three-dimensional attitude angle to generate a second vertex coordinate comprises: perform coordinate transformation on the first vertex coordinate based on the target three-dimensional attitude angle to determine a target vertex coordinate; determine a target distance based on a preset distance range between an image acquisition device and the object to be identified; determine the position information of the imaging plane based on the target distance, and determine the projection point of the target vertex coordinate on the imaging plane based on the target vertex coordinate and the focal point position of the image acquisition device, the position coordinate of the projection point being the second vertex coordinate.

8. A computer device, comprising: comprising: a processor, a memory and a bus, the memory stores machine readable instructions executable by the processor, when the computer device is running, the processor and the memory communicate through the bus, the machine readable instructions are executed by the processor to execute the steps of the image correction method in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, which is executed by the processor to execute the steps of the image correction method in any one of claims 1 to 6.

Images