Calibration method and related device

Abstract

The invention discloses a calibration method and a related device. The method comprises the steps of obtaining a trained calibration scene recognition model and a model use environment; obtaining a corresponding model conversion tool according to the model use environment, and converting the trained calibration scene recognition model by using the model conversion tool; recognizing a scene image corresponding to a to-be-calibrated scene by using the converted and trained calibration scene recognition model to obtain a calibration scene type; and selecting a corresponding calibration algorithm according to the calibration scene type, and performing position calibration in the scene to be calibrated by using the calibration algorithm. In this way, the calibration efficiency and accuracy of object position calibration can be improved.

Description

Technical Field

[0001] The present application relates to the field of position calibration, and in particular to a calibration method and related devices. Background Art

[0002] With the development of intelligent driving technology, surround view technology has been widely adopted to provide intelligent driving chips with more external information, enabling them to respond to the external environment. During the production process, vehicles are required to complete automatic positioning tests, such as automated parking. This automatic positioning process typically requires the use of a calibration cloth, which provides positioning information for the vehicle. To meet user requirements, a variety of calibration cloth styles are generally available. However, during use, the user must manually select the calibration cloth to be used, allowing the computer to calibrate according to the calibration algorithm for the corresponding calibration cloth. This manual selection method is not only time-consuming but also has a high probability of error. Selecting the wrong calibration cloth can lead to calibration errors, further wasting calibration time. Summary of the Invention

[0003] The main purpose of this application is to provide a calibration method and related devices that can improve the calibration efficiency and accuracy of object position calibration.

[0004] The first technical solution adopted in this application is to provide a calibration method. The method includes obtaining a trained calibration scene recognition model and a model usage environment; obtaining a corresponding model conversion tool based on the model usage environment, and using the model conversion tool to convert the trained calibration scene recognition model; using the converted trained calibration scene recognition model to identify a scene image corresponding to a scene to be calibrated to obtain a calibration scene type; selecting a corresponding calibration algorithm based on the calibration scene type, and using the calibration algorithm to perform position calibration in the scene to be calibrated.

[0005] The second technical solution adopted in this application is to provide a calibration device. The calibration device includes an image acquisition module for acquiring a scene image corresponding to the scene to be calibrated; a calibration scene recognition model conversion module for acquiring a trained calibration scene recognition model and a model usage environment, obtaining a corresponding model conversion tool based on the model usage environment, and using the model conversion tool to convert the trained calibration scene recognition model; a calibration scene type recognition module for using the converted trained calibration scene recognition model to identify the scene image corresponding to the scene to be calibrated to obtain a calibration scene type; an algorithm selection module for selecting a corresponding calibration algorithm based on the calibration scene type; and a position calibration module for performing position calibration in the scene to be calibrated using the calibration algorithm.

[0006] The third technical solution adopted by the present application is to provide an electronic device comprising a memory and a processor, wherein the memory is used to store program data, and the program data can be executed by the processor to implement the method described in the first technical solution.

[0007] The fourth technical solution adopted by this application is to provide a computer-readable storage medium / computer program product. The computer-readable storage medium stores program data and can be executed by a processor to implement the method described in the first technical solution. The computer program product includes a computer program. When executed by a processor, the computer program implements the method described in the first technical solution.

[0008] The beneficial effect of this application is that, compared to manual selection, the calibration scene recognition model is used to automatically identify the calibration scene type of the scene to be calibrated, thereby selecting a calibration algorithm based on the obtained calibration scene type, thereby achieving position calibration for the scene to be calibrated, without the need for manual selection of the scene type or the corresponding algorithm. Model recognition also greatly reduces the possibility of algorithm selection errors and improves the efficiency and accuracy of position calibration. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0010] Figure 1 This is a flow chart of the first embodiment of the calibration method of the present application;

[0011] Figure 2 This is a flow chart of the second embodiment of the calibration method of the present application;

[0012] Figure 3 This is a flow chart of the third embodiment of the calibration method of the present application;

[0013] Figure 4 This is a flow chart of the fourth embodiment of the calibration method of the present application;

[0014] Figure 5 This is a flowchart of the fifth embodiment of the calibration method of the present application;

[0015] Figure 6 This is a flow chart of the sixth embodiment of the calibration method of the present application;

[0016] Figure 7 This is a flow chart of a specific embodiment of the calibration method of the present application;

[0017] Figure 8 It is a structural diagram of an embodiment of the calibration device of the present application;

[0018] Figure 9 This is a schematic structural diagram of an embodiment of an electronic device of the present application;

[0019] Figure 10 It is a structural diagram of an embodiment of a computer-readable storage medium of the present application. DETAILED DESCRIPTION

[0020] The following embodiments of the technical solution of the present application will be described in detail with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present application and are therefore only examples and are not intended to limit the scope of protection of the present application.

[0021] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application belongs; the terms used herein are only for the purpose of describing specific embodiments and are not intended to limit this application; the terms "including" and "having" and any variations thereof in the specification and claims of this application and the above-mentioned figure descriptions are intended to cover non-exclusive inclusions.

[0022] References herein to "embodiments" mean that a particular feature, structure, or characteristic described in connection with the embodiments may be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute an independent or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0023] In the description of the embodiments of this application, the technical terms "first" and "second" are used only to distinguish different objects and should not be understood to indicate or imply relative importance or implicitly specify the quantity, specific order, or primary and secondary relationship of the indicated technical features. In the description of the embodiments of this application, the meaning of "plurality" is more than two, unless otherwise clearly and specifically defined.

[0024] In the description of the embodiments of this application, the term "and / or" is simply a description of the association relationship between associated objects, indicating that three relationships can exist. For example, A and / or B can represent the following three situations: A exists alone, A and B exist simultaneously, and B exists alone. In addition, the character " / " in this document generally indicates that the associated objects are in an "or" relationship.

[0025] In the description of the embodiments of the present application, the term "multiple" refers to more than two (including two). Similarly, "multiple groups" refers to more than two groups (including two groups), and "multiple pieces" refers to more than two pieces (including two pieces).

[0026] In the description of the embodiments of the present application, the technical terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise", "axial", "radial", "circumferential", etc., indicating the orientation or position relationship, are based on the orientation or position relationship shown in the accompanying drawings, and are only for the convenience of describing the embodiments of the present application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the embodiments of the present application.

[0027] In the description of the embodiments of the present application, unless otherwise expressly specified or limited, technical terms such as "installed," "connected," "connected," and "fixed" should be understood in a broad sense. For example, they can refer to fixed connections, detachable connections, or integration; mechanical connections or electrical connections; direct connections or indirect connections through an intermediate medium; internal connections between two components or interactions between two components. Those skilled in the art can understand the specific meanings of the above terms in the embodiments of the present application based on specific circumstances.

[0028] In the field of position calibration for intelligent driving, manual selection is often used to select the corresponding calibration cloth for the vehicle during position calibration testing. However, this method is time-consuming and labor-intensive, and has a high probability of error. Therefore, this application chooses to use a recognition model to identify the environment around the vehicle and select the corresponding calibration cloth for the vehicle, eliminating the need for manual selection and speeding up the testing process.

[0029] Since the method of transmitting the image to the server for recognition and then transmitting the result back to the vehicle takes too long and is easily affected by the communication quality of the vehicle's environment, this application chooses to deploy a recognition model locally in the vehicle to recognize the environment around the vehicle.

[0030] Different vehicles may be equipped with different operating systems or use different instruction sets. After designing a recognition model for a certain vehicle's system, the recognition model may not be applicable to the systems installed on other vehicles, resulting in poor recognition results or even inability to perform recognition. However, designing different recognition models for different systems or instruction sets will consume a lot of time and human resources, and when new systems or instruction sets emerge, the model will need to be redesigned. Therefore, this application proposes a calibration method and related devices to solve the above-mentioned problems.

[0031] Reference Figure 1 , Figure 1 This is a flow chart of the first embodiment of the calibration method of this application, which includes but is not limited to the following steps.

[0032] S11: Obtain the trained calibration scene recognition model and the model usage environment.

[0033] After obtaining the trained calibration scene recognition model, because in some cases the model may not be used directly due to different model usage environments, before using the trained calibration scene recognition model to identify scene images and obtain the calibration scene type, it is also necessary to determine the model usage environment.

[0034] Specifically, after obtaining a trained calibrated scene recognition model, it needs to be deployed in the control device of the target object so that the control device of the target object can use the calibrated scene recognition model to determine the scene category. However, the instruction sets supported by the NPU (Neural Network Processing Unit) of different target objects may vary greatly, so the trained calibrated scene recognition model cannot be used directly and needs to be further processed.

[0035] S12: Obtain a corresponding model conversion tool according to the model usage environment, and use the model conversion tool to convert the trained calibration scene recognition model.

[0036] Different NPUs typically have corresponding model conversion tools specifically designed for model conversion. These tools can convert general-purpose models into models suitable for the local usage environment or local instruction set. After obtaining a trained calibrated scene recognition model, use the model conversion tool to convert it into a calibrated scene recognition model that can be used in the current model usage environment.

[0037] For example, the calibration scene recognition model obtained through the above training is a general Keras model. In a certain model usage environment, the NPU-specific model conversion tool is used to convert the general Keras model into a specific model, such as the ATCNN model or the ACTNN model, which can be adapted to the current NPU model usage environment or instruction set.

[0038] S13: Using the converted trained calibration scene recognition model, the scene image corresponding to the calibration scene is recognized to obtain the calibration scene type.

[0039] The scene to be calibrated is the scene in which the user needs to locate the position. The scene image corresponding to the scene to be calibrated is an image obtained by shooting the scene to be calibrated using a shooting device.

[0040] The scenes to be calibrated can be divided into different types according to different user needs.

[0041] In one embodiment, the calibration scene type of the scene to be calibrated is related to the type of calibration cloth included in the scene to be calibrated. For example, if the type of calibration cloth included in the scene to be calibrated is A, the calibration scene type of the scene to be calibrated is determined to be A1; if the type of calibration cloth included in the scene to be calibrated is B, the calibration scene type of the scene to be calibrated is determined to be B1.

[0042] The converted calibration scene recognition model is used to identify the calibration scene type of the scene to be calibrated corresponding to the scene image based on the acquired scene image. When a scene image of the scene to be calibrated that requires position calibration is acquired, it is input into the calibration scene recognition model to obtain the calibration scene type.

[0043] S14: Select a corresponding calibration algorithm according to the calibration scene type, and use the calibration algorithm to perform position calibration in the scene to be calibrated.

[0044] The calibration algorithm is designed to achieve position calibration based on the relevant position information included in the calibration cloth. Different calibration cloth types correspond to different calibration algorithms. For example, if the calibration cloth type is A, the calibration algorithm for type A calibration cloth is A2; if the calibration cloth type is B, the calibration algorithm for type B calibration cloth is B2. The calibration algorithm is pre-designed based on different types of calibration cloths. In this embodiment, after obtaining the calibration scene type, the calibration cloth type corresponding to the calibration scene type can be determined, and the corresponding calibration algorithm can be further determined.

[0045] After selecting the calibration algorithm, execute the selected calibration algorithm to complete the position calibration.

[0046] In this embodiment, this method is applied to a control device mounted on a target object. Scene images are captured by a camera mounted on the target object. During position calibration, the control device controls the position of the target object within the scene to be calibrated according to a calibration algorithm. The target object can be a mobile device such as a vehicle or aircraft.

[0047] In this embodiment, compared with manual selection, the calibration scene recognition model is used to automatically identify the calibration scene type of the scene to be calibrated, so that the calibration algorithm is selected according to the obtained calibration scene type, thereby achieving position calibration of the scene to be calibrated, without the need to manually select the scene type of the scene to be calibrated or the corresponding algorithm. Model recognition also greatly reduces the possibility of error in algorithm selection and improves the efficiency and accuracy of position calibration. In addition, the present application obtains a trained calibration scene recognition model and a model usage environment, and the model conversion tool obtained based on the model usage environment can convert the calibration scene recognition model so that it is suitable for the corresponding model usage environment, thereby improving the universality of the scene recognition model, improving the stability and accuracy of scene type recognition, thereby improving the selection accuracy of the calibration algorithm, and improving the calibration efficiency and accuracy of position calibration.

[0048] Reference Figure 2 , Figure 2 This is a flow chart of the second embodiment of the calibration method of this application. This method further defines the calibration scene recognition model training method. It includes but is not limited to the following steps.

[0049] S21: Obtain training scene images and corresponding classification labels.

[0050] The training scene images are scene images used for training the calibration recognition model. The training scene images can be acquired accordingly according to the needs of the user. For example, the user needs to identify the calibration scene type A1 and the calibration scene type B1. Then, when acquiring the training images, scene images including the calibration cloth A under different environmental parameters and scene images including the calibration cloth B under different environmental parameters are acquired. Environmental parameters may include brightness, contrast, saturation, sharpness, exposure, gain, day and night parameter conversion, backlight compensation, white balance and noise reduction, etc. The wider the range of the environmental parameters used, the better the recognition adaptability of the trained calibration scene recognition model to scene images acquired in different environments.

[0051] The classification label is a label that identifies the type of the training scene image. Each training scene image has a corresponding classification label.

[0052] S22: Convert the training scene image into array form and convert the classification label into one-hot encoding form.

[0053] An array stores a fixed-size sequential collection of elements of the same type. An array can be considered a collection of variables of the same type. One-hot encoding is a method of processing classification problems in machine learning. It creates a new binary feature for each possible value of each classification feature. The training data is converted into array form, and the classification labels are converted into one-hot encoding form. These two forms are suitable for neural network training and facilitate subsequent calculations in the calibration scene recognition model. Data processing in the calibration scene recognition model is usually matrix data processing, and it is more convenient to perform operations in array form. The calibration scene recognition model involves multi-classification problems. When the calibration scene recognition model finally outputs the result with the softmax activation function, the label encoded in one-hot encoding form can be directly compared with the output result, thereby improving computational efficiency.

[0054] In one embodiment, after obtaining the training scene images, all the training scene images are converted into NumPy array format, and the corresponding classification labels are converted into one-hot encoding vectors.

[0055] S23: Using the converted training scene images and the corresponding classification labels to train a pre-built calibration scene recognition model to obtain a trained calibration scene recognition model.

[0056] The converted training scene images and classification labels are input into the calibration scene recognition model for training.

[0057] In one embodiment, the model is trained using the Adam optimization algorithm.

[0058] In one embodiment, the loss function used in the training process is specified to be a cross entropy loss function ategorical_crossentropy. A ategorical_crossentropy loss function is a commonly used loss function in multi-classification problems and is suitable for cases where the class labels are one-hot encoded.

[0059] After the training is completed, a trained calibration scene recognition model is obtained.

[0060] In one embodiment, the pre-built calibration scene recognition model includes a convolutional layer, a maximum pooling layer, a flattening layer, a fully connected layer, and a dropout layer. The convolutional layer is used to extract data features. The maximum pooling layer is used to reduce spatial dimensions while retaining distinct features. The flattening layer is used to reduce dimensionality. The fully connected layer is used to separate individual features. The dropout layer is used to prevent overfitting.

[0061] Specifically, you can first create a Keras Sequential model class, which can be directly added to create a model.

[0062] Then we start adding it to the Sequential Model class. The first layer is Conv2D, a two-dimensional convolutional layer used to extract spatial features of the image.

[0063] The second layer is MaxPooling2D, which is a two-dimensional maximum pooling layer used to reduce the spatial dimension of the features while retaining the most important information.

[0064] The third layer is still Conv2D, which is the second convolutional layer. The structure is similar to the first one, but this time more filters are learned to further extract more complex features.

[0065] A pooling layer is added to the fourth layer to further reduce the spatial dimension of the features.

[0066] The fifth layer is Flatten, which mainly converts the multi-dimensional input into one dimension and is used to convert the multi-dimensional feature maps output by the convolution layer and the pooling layer into vectors for input into the fully connected layer.

[0067] The sixth layer is Dense, which is a fully connected layer. Each of its nodes is connected to all nodes in the previous layer.

[0068] The seventh layer is Dropout, which is used to prevent overfitting.

[0069] The eighth layer is another fully connected layer. The number of neurons in the eighth layer corresponds to the type of calibration scene being output. If the trained model can recognize six different calibration scene types, the eighth fully connected layer has six neurons. If the trained model can recognize eight different calibration scene types, the eighth fully connected layer has eight neurons. This fully connected layer is followed by a softmax activation function, which outputs the calibration scene type with the highest probability.

[0070] Reference Figure 3 , Figure 3 This is a flow chart of the third embodiment of the calibration method of this application. This method is a further extension of the above embodiment, and includes but is not limited to the following steps.

[0071] S31: Determine whether the version of the calibrated scene recognition model on the local side is consistent with the version of the calibrated scene recognition model on the server side.

[0072] When the local client needs to use the calibration scene recognition model, it also needs to determine whether the calibration scene recognition model on the server has been updated before obtaining the trained calibration scene recognition model. Based on the determination result, the local client can determine whether the calibration scene recognition model has been updated. This determination can be made before each use of the calibration scene recognition model, or at a preset time interval. If the versions are consistent, proceed to step S32. If not, proceed to step S33.

[0073] Specifically, the determination can be made by obtaining the version number of the local calibration scene recognition model and the version number of the server calibration scene recognition model.

[0074] S32: Obtain a local calibration scene recognition model.

[0075] If the versions are consistent, it means that the calibrated scene recognition model on the server has not been updated. In this case, the calibrated scene recognition model on the local side is directly obtained for scene image recognition.

[0076] S33: Obtain the calibrated scene recognition model on the server side.

[0077] Version inconsistency indicates that the server-side calibrated scene recognition model has been updated. The local end needs to obtain the server-side calibrated scene recognition model for scene image recognition.

[0078] In one embodiment, after obtaining the trained calibration scene recognition model, the method includes: performing pruning and / or quantization operations on the trained calibration scene recognition model.

[0079] Due to the limited processing power of the target object's processor, the calibrated scene recognition model needs to be simplified before deploying it. This simplification process can include pruning and / or quantization. This simplification process can be performed before or after model conversion, but is generally more convenient before conversion.

[0080] Pruning refers to the removal of unimportant or redundant weight connections or neurons in a neural network. This is typically accomplished by setting a threshold and then pruning connections whose absolute weights are smaller than the threshold. This approach can achieve the following benefits, particularly in embedded products: It reduces model size. By removing unimportant weights, the number of model parameters can be significantly reduced, thereby reducing the model's storage footprint. It also accelerates inference speed: Reducing the number of parameters means less computation is required during model inference, speeding up the model's computational speed.

[0081] Quantization converts floating-point weights and activation values in a neural network into integers with lower bit widths, for example, converting 32-bit floating-point numbers to 8-bit integers. It primarily reduces storage and bandwidth requirements, with quantized models occupying less storage space and reducing the amount of data transferred between memory and the device. It also improves computational efficiency, as integer operations are generally faster than floating-point operations. Therefore, quantized models can achieve faster inference speeds on hardware that doesn't support floating-point operations or has slower floating-point operations.

[0082] The calibration scene recognition stored in the server is usually a general model that is convenient for deployment on the local side of each target object. The general model facilitates adaptive conversion on the local side of each target object. Therefore, after obtaining the calibration scene recognition model of the server side on the local side, the obtained calibration scene recognition model can be simplified according to the actual processing performance or needs. If the calibration scene recognition model versions of the local side and the server side are consistent, when the calibration scene recognition model of the local side used for comparison is a model that has been converted and deployed on the local side, there is no need to simplify it. When the calibration scene recognition model of the local side used for comparison is a model that has not yet been deployed on the local side, simplification and / or conversion processing is still required.

[0083] The simplification and conversion processing will not affect the version number of the calibrated scene recognition model. The calibrated scene recognition model that has been simplified and / or converted on the local side maintains the original version number.

[0084] Reference Figure 4 , Figure 4 This is a flow chart of the fourth embodiment of the calibration method of this application. This method is a further extension of the above embodiment. It includes but is not limited to the following steps.

[0085] S41: performing data conversion on the scene image according to the training data format of the calibrated scene recognition model.

[0086] When using the converted trained calibration scene recognition model to identify the scene image of the scene to be calibrated and obtain the calibration scene type, the obtained scene image can be preprocessed to improve the recognition efficiency and accuracy of the scene recognition model. Preprocessing can include converting the scene image format to the same data format as the training scene image for the calibration scene recognition model. The training data format is the data format of the training scene image.

[0087] For example, when training the model, the training scene image is a grayscale image with a size of 32×32. After acquiring the scene image, it is also converted to a grayscale image with a size of 32×32 data format.

[0088] S42: Input the converted scene image into a calibration scene recognition model for recognition to obtain a calibration scene type.

[0089] The converted scene images in the same data format are input into the calibrated scene recognition model for recognition, making the results more accurate.

[0090] Because the scene images for training the model are ultimately limited, but the actual calibration scenes are varied, it is very likely that recognition errors will still occur in practice. In this case, further settings can be made to improve the processing capabilities of the calibration scene recognition model.

[0091] In actual scenarios, position calibration fails in the following two situations:

[0092] 1. Automatic identification and calibration errors may result in the selection of an incorrect calibration algorithm, leading to calibration failure.

[0093] 2. The automatic recognition and calibration layout is correct, but the calibration algorithm cannot correctly calibrate the current scene.

[0094] The second case is not considered in this application. This application is only set up for the first case of recognition error.

[0095] Reference Figure 5 , Figure 5 This is a flow chart of the fifth embodiment of the calibration method of this application. This method further defines the above embodiment and includes but is not limited to the following steps.

[0096] S51: In response to position calibration failure, obtaining a manual calibration scene type.

[0097] After position calibration is performed in the scene to be calibrated using the calibration algorithm, if the position calibration fails, a manual selection interface may be provided for the user to manually select the calibration scene type.

[0098] S52: Select a corresponding calibration algorithm according to the manual calibration scene type.

[0099] After the selection is completed, similar to the above embodiment, the calibration algorithm is determined according to the manual calibration scene type.

[0100] S53: Perform position calibration in the scene to be calibrated using a calibration algorithm corresponding to the manual calibration scene type.

[0101] The position calibration is performed again according to the determined calibration algorithm.

[0102] In this embodiment, the failure of position calibration indicates that the calibration scene recognition model has made an error in its recognition and selected the wrong calibration scene type, which in turn led to the selection of the wrong calibration algorithm, causing position calibration failure. Therefore, the manually selected calibration scene type is re-obtained and position calibration is performed again.

[0103] Reference Figure 6 , Figure 6 This is a flow chart of the sixth embodiment of the calibration method of this application. This method further defines the above embodiment and includes but is not limited to the following steps.

[0104] S61: In response to the position calibration being successful, the scene image and the corresponding manually calibrated scene type are determined as a new training scene image and a corresponding classification label.

[0105] After performing position calibration in the scene to be calibrated using the calibration algorithm corresponding to the manual calibration scene type, if the position calibration fails, it means that the calibration algorithm cannot correctly calibrate the current scene. This situation is not considered for now. If the position calibration is successful, it means that the calibration scene type previously identified by the calibration recognition model is incorrect, and the manually selected manual calibration scene type is correct. Therefore, these scene images that the model incorrectly recognized are determined as new training scene images, and the manually selected manual calibration scene type is determined as the corresponding classification label for these scene images.

[0106] S62: Train the calibration scene recognition model using the new training scene images and corresponding classification labels.

[0107] The calibrated scene recognition model is retrained using these new training scene images and corresponding classification labels.

[0108] In one embodiment, model training and generation are performed on a server. After determining a new training scene image and its corresponding classification label, the target object is uploaded to the server. The server is provided with a database of uploaded training scene images. When the number of scene images in the database reaches a first preset threshold, the scene images in the database are integrated with the basic training data to generate new training data for retraining the calibration scene recognition model.

[0109] Set up an uploaded training scene image database to temporarily store new training scene images, and set a preset threshold so that data integration can be performed only after the number of new training scene images reaches a certain level, avoiding frequent updates of the training set and wasting computing resources.

[0110] In one embodiment, the uploaded training scene image database also includes multiple sub-databases corresponding to different classification labels. When the number of scene images in a sub-database reaches a second preset threshold, the scene images in that sub-database can be individually integrated with the basic training data to generate new training data. The second preset threshold value set for each sub-database is smaller than the first preset threshold value for the entire uploaded training scene image database.

[0111] S63: The trained calibration scene recognition model is further used for recognition of calibration scene types.

[0112] The retrained calibration scene recognition model replaces the original calibration scene recognition model and is deployed on the control device of the target object to continue to be used for the recognition of the calibration scene type.

[0113] In one embodiment, the training and generation of the model are performed in the server, and the control device of the target object obtains the calibrated scene recognition model from the server for scene recognition. The server generates a corresponding index when training the generated model, and each calibrated scene recognition model obtained through training has its corresponding index to represent its training version. Before acquiring the scene image and using the trained calibrated scene recognition model for recognition, the control device of the target object sends an acquisition message to the server to obtain the index of the training version of the server's current latest model, and compares the obtained index with the index of the calibrated scene recognition model deployed by itself. If they are inconsistent, the calibrated scene recognition model is re-acquired from the server for recognition. If they are consistent, the subsequent recognition steps are performed using its own calibrated scene recognition model.

[0114] The following is a specific example to illustrate the calibration method of this application in more detail. Figure 7 , Figure 7 This is a flow chart of a specific embodiment of the calibration method of this application.

[0115] First, in the server, the pre-built model is trained using the training scene images and corresponding classification labels, and the obtained model is saved.

[0116] After the camera captures an image of the scene to be calibrated, it first checks whether the calibration scene recognition model on the server has been updated. If so, it downloads the new calibration scene recognition model for deployment and subsequent recognition. If no update exists, subsequent recognition continues using the deployed calibration scene recognition model. A calibration algorithm is selected based on the identified calibration scene type, and position calibration is performed. If calibration succeeds, the process ends. If calibration fails, the camera enters manual scene type selection mode. After obtaining the manually selected calibration scene type, the corresponding calibration algorithm is selected and position calibration continues. If calibration still fails, it indicates that the calibration algorithm does not support the current scene, a calibration error message is displayed, and the process ends. If calibration succeeds, it indicates that the recognition capability of the calibration scene recognition model needs improvement. The scene image with the incorrect recognition is used as a new training scene image, and the manually selected calibration scene type is uploaded to the server as the corresponding classification label. After the server detects that the uploaded new training scene image has reached a certain amount of data, it integrates it with the basic training data to generate new data, and retrains the calibration scene recognition model stored on the server to achieve an update.

[0117] Reference Figure 8 , Figure 8 This is a structural diagram of an embodiment of the calibration device of the present application.

[0118] The calibration device includes an image acquisition module 10 , a calibration scene type recognition module 20 , an algorithm selection module 30 and a position calibration module 40 .

[0119] The image acquisition module 10 is used to acquire a scene image corresponding to the scene to be calibrated.

[0120] The calibration scene recognition model conversion module 20 is used to obtain the trained calibration scene recognition model and the model usage environment, obtain the corresponding model conversion tool according to the model usage environment, and use the model conversion tool to convert the trained calibration scene recognition model.

[0121] The calibration scene type recognition module 30 is deployed with a converted and trained calibration scene recognition model. The module uses the converted and trained calibration scene recognition model to recognize the scene image and obtain the calibration scene type.

[0122] The algorithm selection module 40 is used to select a corresponding calibration algorithm according to the calibration scene type.

[0123] The position calibration module 50 is used to perform position calibration in the scene to be calibrated using a calibration algorithm.

[0124] In one embodiment, the calibration scene recognition model is trained in the following manner, including: obtaining training scene images and corresponding classification labels; converting the training scene images into array form, and converting the classification labels into one-hot encoding form; using the converted training scene images and corresponding classification labels to train a pre-built calibration scene recognition model to obtain a trained calibration scene recognition model.

[0125] In one embodiment, obtaining a trained calibration scene recognition model includes: determining whether the version of the calibration scene recognition model on the local side is consistent with the version of the calibration scene recognition model on the server side; if the versions are consistent, obtaining the calibration scene recognition model on the local side; if the versions are inconsistent, obtaining the calibration scene recognition model on the server side.

[0126] In one embodiment, after obtaining the trained calibration scene recognition model, the method includes: performing pruning and / or quantization operations on the trained calibration scene recognition model.

[0127] In one embodiment, the converted trained calibration scene recognition model is used to identify the scene image corresponding to the scene to be calibrated to obtain the calibration scene type, including: performing data conversion on the scene image according to the training data format of the calibration scene recognition model; inputting the converted scene image into the calibration scene recognition model for identification to obtain the calibration scene type.

[0128] In one embodiment, after position calibration is performed in the scene to be calibrated using a calibration algorithm, the method includes: in response to a failure of position calibration, obtaining a manual calibration scene type; selecting a corresponding calibration algorithm according to the manual calibration scene type; and performing position calibration in the scene to be calibrated using the calibration algorithm corresponding to the manual calibration scene type.

[0129] In one embodiment, after position calibration is performed in the scene to be calibrated using a calibration algorithm corresponding to the manually calibrated scene type, the method includes: in response to successful position calibration, determining the scene image and the corresponding manually calibrated scene type as a new training scene image and a corresponding classification label; using the new training scene image and the corresponding classification label to train a calibration scene recognition model; and continuing to use the trained calibration scene recognition model for recognition of calibration scene types.

[0130] The same or similar steps can be referred to the description in the above embodiments and will not be repeated here.

[0131] like Figure 9 As shown, Figure 9 This is a structural diagram of an embodiment of an electronic device of the present application.

[0132] The electronic device includes a processor 110 and a memory 120 .

[0133] The processor 110 controls the operation of the electronic device and may also be referred to as a CPU (Central Processing Unit). The processor 110 may be an integrated circuit chip with the ability to process signal sequences. The processor 110 may also be a general-purpose processor, a digital signal sequence processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component. The general-purpose processor may be a microprocessor or any conventional processor.

[0134] The memory 120 stores instructions and program data required for the processor 110 to operate.

[0135] The processor 110 is configured to execute instructions to implement the method provided by any one of the embodiments and possible combinations of the aforementioned calibration methods of the present application.

[0136] like Figure 10 As shown, Figure 10 This is a structural diagram of an embodiment of a computer-readable storage medium of the present application.

[0137] An embodiment of the computer-readable storage medium of the present application includes a memory 210, which stores computer-executable instructions. When the computer-executable instructions are executed, the method provided by any embodiment and possible combination of the calibration method of the present application is implemented.

[0138] The memory 210 may include a medium that can store program instructions, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, or it may be a server that stores the program instructions. The server may send the stored program instructions to other devices for execution, or it may execute the stored program instructions itself.

[0139] The present application also provides a computer program product. The computer program product includes a computer program. When the computer program is executed, it implements the device startup method provided by any embodiment and possible combination of the calibration method described above.

[0140] In summary, compared to manual selection, the calibration scene recognition model is used to automatically identify the calibration scene type of the scene to be calibrated, and then the calibration algorithm is selected according to the obtained calibration scene type, thereby achieving position calibration of the scene to be calibrated, without the need to manually select the scene type of the scene to be calibrated or the corresponding algorithm. Model recognition also greatly reduces the possibility of algorithm selection errors and improves the efficiency and accuracy of position calibration. In addition, the present application obtains a trained calibration scene recognition model and a model usage environment, and the model conversion tool obtained based on the model usage environment can convert the calibration scene recognition model so that it is suitable for the corresponding model usage environment, thereby improving the universality of the scene recognition model, improving the stability and accuracy of scene type recognition, thereby improving the selection accuracy of the calibration algorithm, and improving the calibration efficiency and accuracy of position calibration.

[0141] In the several embodiments provided in this application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the modules or units is merely a logical functional division. In actual implementation, other division methods may be used, such as combining or integrating multiple units or components into another system, or ignoring or not implementing certain features.

[0142] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of this embodiment.

[0143] In addition, each functional unit in each embodiment of the present application may be integrated into a processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The above-mentioned integrated units may be implemented in the form of hardware or software functional units.

[0144] If the integrated units in the above other embodiments are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0145] The above description is merely an embodiment of the present application and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A calibration method, characterized in that: The method comprises: Obtain the trained calibration scene recognition model and the model usage environment; Obtaining a corresponding model conversion tool according to the model usage environment, and using the model conversion tool to convert the trained calibration scene recognition model; Using the converted trained calibration scene recognition model to identify the scene image corresponding to the scene to be calibrated, to obtain the calibration scene type; A corresponding calibration algorithm is selected according to the calibration scene type, and the calibration algorithm is used to perform position calibration in the scene to be calibrated.

2. The method according to claim 1, characterized in that The calibration scene recognition model is trained in the following manner, including: Get training scene images and corresponding classification labels; Convert the training scene image into an array form, and convert the classification label into a one-hot encoding form; The pre-built calibration scene recognition model is trained using the converted training scene images and the corresponding classification labels to obtain the trained calibration scene recognition model.

3. The method according to claim 1, characterized in that The step of obtaining a trained calibration scene recognition model includes: Determining whether the version of the calibrated scene recognition model on the local side is consistent with the version of the calibrated scene recognition model on the server side; If the versions are consistent, the calibrated scene recognition model on the local side is obtained; if the versions are inconsistent, the calibrated scene recognition model on the server side is obtained.

4. The method according to claim 1, wherein After obtaining the trained calibration scene recognition model, the method includes: Pruning and / or quantization operations are performed on the trained calibration scene recognition model.

5. The method according to claim 1, wherein The converted trained calibration scene recognition model is used to identify the scene image corresponding to the calibration scene to obtain the calibration scene type, including: Performing data conversion on the scene image according to the training data format of the calibrated scene recognition model; The converted scene image is input into the calibration scene recognition model for recognition to obtain the calibration scene type.

6. The method according to claim 1, characterized in that After performing position calibration in the scene to be calibrated by using the calibration algorithm, the method includes: In response to a position calibration failure, obtaining a manual calibration scene type; Selecting the corresponding calibration algorithm according to the manual calibration scene type; Position calibration is performed in the scene to be calibrated using the calibration algorithm corresponding to the manual calibration scene type.

7. The method according to claim 6, characterized in that After performing position calibration in the scene to be calibrated using the calibration algorithm corresponding to the manual calibration scene type, the method includes: In response to successful position calibration, determining the scene image and the corresponding manually calibrated scene type as a new training scene image and a corresponding classification label; Training the calibration scene recognition model using the new training scene image and the corresponding classification label; The trained calibration scene recognition model is further used for recognizing the calibration scene type.

8. A calibration device, characterized in that: include: An image acquisition module is used to acquire a scene image corresponding to the scene to be calibrated; A calibration scene recognition model conversion module is used to obtain a trained calibration scene recognition model and a model usage environment, obtain a corresponding model conversion tool according to the model usage environment, and use the model conversion tool to convert the trained calibration scene recognition model; a calibration scene type recognition module, configured to use the converted trained calibration scene recognition model to recognize the scene image corresponding to the scene to be calibrated, and obtain a calibration scene type; An algorithm selection module, configured to select a corresponding calibration algorithm according to the calibration scene type; The position calibration module is used to perform position calibration in the scene to be calibrated using the calibration algorithm.

9. An electronic device, characterized in that: The system comprises a memory and a processor, wherein the memory is used to store program data, and the program data can be executed by the processor to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium / computer program product, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 7 when executed by a processor; The computer program product comprises a computer program, which implements the method according to any one of claims 1 to 7 when executed by a processor.

Images