Model training method, core adjusting method and device, medium and electronic equipment

Abstract

The embodiment of the invention provides a model training method, a core adjusting method and device, a medium and electronic equipment. The model training method comprises the following steps: acquiring a first image of a target object; obtaining a sample image based on the first image; a training sample data set is obtained based on the sample image, the training sample data set comprises a second image, and the second image is an image obtained after the first image passes through the optical system; and training an initial model through the sample data set until the initial model meets a preset condition, and obtaining a target model. According to the method, the core of the to-be-core-adjusted model can be accurately adjusted through the target model.

Description

Technical Field

[0001] The present application belongs to the field of artificial intelligence technology and relates to the field of optical system assembly technology, and in particular to a model training method, core adjustment method, device, medium and electronic equipment. Background Art

[0002] With the advancement of science and technology, optical lenses are widely used, but their performance is significantly affected by processing and assembly errors. Assembly errors (such as eccentricity, tilt, and spacing errors) will lead to aberrations and reduced resolution, affecting image quality. Early adjustment relied on mechanical adjustment and empirical judgment, with low precision and poor efficiency; the later use of high-precision sensors (such as lasers and wavefront sensors) improved accuracy, but the cost was high and the equipment was complex, making it difficult to popularize. Existing intelligent core adjustment methods use measured data to train machine learning models to predict the core adjustment position, but this requires a large amount of data collection, which is time-consuming and labor-intensive, especially when adjusting multiple groups of lenses. It is more cumbersome and inefficient. Summary of the invention

[0003] The embodiments of the present application provide a model training method, a core tuning method, a device, a medium and an electronic device to solve the technical problems of long manual core tuning time and poor core tuning quality existing in traditional technologies.

[0004] A first aspect of an embodiment of the present application provides a model training method, which is applied to an electronic device, wherein the electronic device is communicatively connected to an optical system, and the method comprises: acquiring a first image of a target object; obtaining a sample image based on the first image; obtaining a training sample data set based on the sample image, wherein the training sample data set comprises a second image, and the second image is an image obtained after the first image passes through the optical system; and training an initial model through the sample data set until the initial model meets a preset condition, thereby obtaining a target model.

[0005] In some possible implementations of the present application, obtaining the sample image based on the first image includes: setting image information of the first image according to parameters of the optical system; and obtaining the sample image based on the image information and the parameters of the first image.

[0006] In some possible implementations of the present application, obtaining a training sample data set based on the sample image further includes: obtaining the second image based on the sample image and the optical system; and obtaining a training sample data set based on an error of the optical system and the second image.

[0007] In some possible embodiments of the present application, the error in the optical system includes a first error and a second error, wherein the first error includes a processing error corresponding to the relay lens in the optical system and / or an assembly error corresponding to the relay lens, a processing and assembly error corresponding to the non-aligned lens group in the lens to be aligned, and a processing error of the aligned lens group; the second error includes an assembly error corresponding to the aligned lens group in the lens to be aligned.

[0008] In some possible implementations of the present application, the method further includes: classifying the data in the training sample data set according to the type of the error.

[0009] In some possible implementations of the present application, the image information of the first image includes image dimension, image height and image size.

[0010] A second aspect of an embodiment of the present application provides a centering method, the method comprising: acquiring a target image; inputting the target image into a pre-trained target model to obtain a misalignment amount corresponding to the centering lens group, wherein the target model is obtained according to the above-mentioned model training method; after controlling the movement of the mechanical motion structure according to the misalignment amount, acquiring the target image again; and determining whether the centering of the lens to be aligned corresponding to the centering lens group is completed according to the quality of the target image acquired again.

[0011] A third aspect of an embodiment of the present application provides a model training device, which includes: an acquisition module for acquiring a first image of a target object; a processing module for obtaining a sample image based on the first image; the processing module is also used to obtain a training sample data set based on the sample image, wherein the training sample data set includes a second image, which is an image obtained after the first image passes through the optical system; the processing module is also used to train an initial model through the sample data set until the initial model meets preset conditions to obtain a target model.

[0012] A fourth aspect of an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores at least one instruction, and when the at least one instruction is executed by a processor, the core adjustment method as described above is implemented.

[0013] A fifth aspect of an embodiment of the present application provides an electronic device, comprising: a memory, and a processor, wherein the processor executes computer-readable instructions stored in the memory to implement the core adjustment method.

[0014] The core adjustment method provided in the embodiment of the present application obtains a first image of the target object; obtains a sample image based on the first image; obtains a training sample data set based on the sample image, wherein the training sample data set includes a second image, and the second image is an image obtained after the first image passes through the optical system; trains an initial model through the sample data set until the initial model meets preset conditions to obtain a target model; and then uses the template model to perform core adjustment. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0016] Figure 1 A schematic diagram of an application environment for a model training method provided in an embodiment of the present application.

[0017] Figure 2 A schematic structural diagram of a six-degree-of-freedom translation platform provided in an embodiment of the present application.

[0018] Figure 3 A flowchart of the model training method provided in an embodiment of the present application.

[0019] Figure 4 The error combination diagram required for the training sample data set provided in the embodiment of the present application.

[0020] Figure 5 A schematic flow chart of the core tuning method provided in an embodiment of the present application.

[0021] Figure 6 A functional block diagram of a model training device provided in an embodiment of the present application.

[0022] Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0023] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application is described in detail below with reference to the accompanying drawings and specific embodiments.

[0024] It should be noted that in this application, "at least one" means one or more, and "more than one" means two or more than two. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The terms "first", "second", "third", "fourth", etc. (if any) in the specification, claims and drawings of this application are used to distinguish similar objects, rather than to describe a specific order or sequence.

[0025] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a concrete way. The following embodiments and features in the embodiments may be combined with each other without conflict.

[0026] With the rapid advancement of science and technology, optical lenses are increasingly used in various fields, and the requirements for their performance are also rising. However, the performance of optical lenses is not only affected by the optical design, but also greatly restricted by the errors in the processing and assembly process. At present, although many lenses have reached a fairly high imaging quality standard in design, in actual applications, due to processing and assembly errors, their imaging effects often fail to achieve the expected ideal state. Therefore, how to effectively solve the processing and assembly errors of lenses has become the key to improving the performance of optical lenses.

[0027] During the lens assembly process, various assembly errors such as eccentricity, tilt and spacing errors between lenses are difficult to avoid. These errors will not only cause additional aberrations in the optical system, but also significantly reduce the resolution of the lens, thereby affecting the image quality. In practical applications, this reduction in image quality may affect the performance of the product and even determine its success or failure.

[0028] Early optical system adjustment mainly relied on mechanical adjustment and empirical judgment, which is not only of limited accuracy but also inefficient. Although adjustment solutions using high-precision sensors such as lasers and wavefront sensors have emerged, which can significantly improve the centering accuracy, the high hardware cost and complex device structure limit their widespread application. Although there are some devices on the market that can traverse and find the best adjustment position, these devices often take a long time and may not even be as fast as manual adjustment, which obviously cannot meet the current huge demand for high-quality lenses. Therefore, it is necessary to find an adjustment method that can ensure high accuracy and improve efficiency.

[0029] With the development of optical technology and intelligent algorithms, the alignment accuracy and efficiency of lens modules are of vital importance to the improvement of imaging quality. In the prior art, the lens alignment can be performed by using measured data to obtain a data set to train an intelligent alignment method. This method adds assembly errors to the lens module to be aligned, collects data such as the modulation transfer function (MTF) curve and aberration characteristics of the lens, trains neural networks or other machine learning models, and realizes intelligent prediction of the alignment position. However, this intelligent alignment method based on measured data training still has certain limitations in practical applications. For each lens module to be aligned, a large amount of data collection work is required to generate the corresponding data set in order to achieve accurate prediction of the alignment position. This data collection process not only consumes time and resources, but also directly affects the measurement efficiency of the production line.

[0030] Especially when facing the core adjustment of multiple groups of lenses, the problem becomes more complicated. Multiple groups of lenses need to adjust the position of each lens group separately, and the relative position relationship between the lens groups must also be considered comprehensively. In this case, the method of collecting data one by one to train the model will be particularly cumbersome and inefficient, further increasing the production cost and time cost. In addition, during the core adjustment of multiple groups of lenses, the differences in the characteristics of different lens groups make the unified processing of data and the adaptability of the model face higher challenges. Therefore, how to reduce the collection needs for single or multiple groups that need to be adjusted, simplify the data collection process, and at the same time improve the core adjustment efficiency and measurement accuracy has become an important issue that needs to be urgently solved in this field.

[0031] In order to solve the above problems, this application provides a model training method and a core tuning method, which pre-trains the target model and then uses the target model to perform core tuning. Figure 1 , is a schematic diagram of an application environment of a core adjustment method provided in an embodiment of the present application. The core adjustment method is applied to an optical system. Figure 1 As shown, the optical system 100 includes a first image 1, a relay lens 2, and a lens to be aligned. 3, alignment lens group 4, image sensor 5, second image 6 and mechanical motion structure 7. The optical system 100 is connected to the electronic device 200 for communication. For example, the image sensor 5 and the mechanical motion structure 7 in the optical system 100 are connected to the electronic device 200 for communication.

[0032] In some embodiments of the present application, the first image 1 is an object-side pattern, which is an image corresponding to a target object, for example, an image obtained by photographing the target object with a camera.

[0033] In some embodiments of the present application, the lens 3 to be adjusted can be an infinite conjugate lens, a finite conjugate lens or an incomplete lens. The adjusting lens group 4 is the first part of the lens 3 to be adjusted. When the lens 3 to be adjusted is an infinite conjugate lens, a relay lens 2 needs to be added to the optical system, and the first image 1 is located on the working surface of the relay lens 2. Since the design of the infinite conjugate lens is suitable for incident parallel light, the relay lens 2 is required to convert the object pattern into parallel light to make it suitable for the imaging requirements of the infinite conjugate lens. When the lens 3 to be adjusted is a finite conjugate lens, the relay lens 2 is not required to be added to the optical system, and the first image 1 is located on the object plane of the lens 3 to be adjusted. Since the finite conjugate lens can directly image the first image 1, the relay lens 2 is not required. The first image 1 can be directly located on the object plane of the lens 3 to be adjusted. When the second lens 3 is an incomplete lens, a compensating lens needs to be added to the optical system, and the first image 1 is located on the working surface of the compensating lens. When the second lens is an incomplete lens, it lacks certain optical functions and cannot form an image, so a compensating lens needs to be added to complete the optical path, and the role of the relay lens 2 is replaced by the compensating lens.

[0034] In some embodiments of the present application, the lens 3 to be aligned includes a non-aligned lens group and an aligned lens group, wherein the non-aligned lens group is an assembled fixed part, and the aligned lens group is an incompletely assembled lens or lens group to be aligned. The non-aligned lens group and the lens or lens group to be aligned are combined to form a lens, and a complete lens can form an image.

[0035] In some embodiments of the present application, the centering lens group 4 is located inside the lens 3 to be aligned, and is connected to the mechanical motion structure 7 . The position of the centering lens group 4 can be adjusted by controlling the movement of the mechanical motion structure 7 .

[0036] In some embodiments of the present application, the image sensor 5 is used to capture the second image after the lens 3 to be aligned is imaged, and send the second image to the electronic device 200. The electronic device 200 analyzes the image quality according to the received second image, and determines the performance of the lens 3 to be aligned according to the image quality. The mechanical motion structure 7 can be a six-degree-of-freedom displacement stage. Figure 2 The six-degree-of-freedom translation platform has six degrees of freedom: roll, pitch, yaw, lift, thrust, and lateral movement. Among them, roll, pitch, and yaw are the degrees of freedom of angular rotation around the x, y, and z directions. Lift, thrust, and lateral movement are the degrees of freedom of axial displacement in the x, y, and z directions. In this embodiment, a high-precision six-degree-of-freedom translation platform in parallel can be used to control the displacement and angle of the core lens 3 to be adjusted in the x, y, and z directions respectively.

[0037] According to actual needs, the optical system 100 may also include other auxiliary optical elements, such as filters, etc., to further improve system performance. Figure 1 Not shown in detail.

[0038] In some possible scenarios, the electronic device 200 may also be connected to the image sensor 5 and / or the mechanical motion structure 7 through a network. The network may be a wired network communication or a wireless network communication. The wired network may be any one of a local area network, a metropolitan area network, and a wide area network, a wireless fidelity (Wireless Fidelity, Wi-Fi), a self-organizing network wireless communication (ZigBee Wireless Networks, ZigBee) technology, an ultra-wideband (Ultra Wideband, UWB) technology, a wireless universal serial bus (Universal Serial Bus, USB), and the like.

[0039] Figure 3 is a flow chart of the core adjustment method provided in the embodiment of the present application, such as Figure 3 As shown, the core adjustment method is applied in electronic equipment. According to different requirements, the order of the steps in the flow chart can be changed, and some steps can be omitted.

[0040] Step S1, acquiring a first image of a target object.

[0041] Step S2: obtaining a sample image based on the first image.

[0042] In an embodiment of the present application, obtaining a sample image based on a first image includes: setting image information of the first image according to parameters of an optical system; obtaining a sample image based on the image information and parameters of the first image. Specifically, the image information of the first image is set according to the magnification, resolution and centering field of view of the optical system. Among them, the image information of the first image includes image size, image height and image size. After setting the image information of the first image, a first image of a preset size, preset height and preset size can be obtained, and N sample images are obtained according to the parameters of the set first image. Among them, the parameters include image uniformity and installation error, etc. For example, N sample images a1, a2...an are obtained according to different image uniformities and installation errors.

[0043] In the embodiment of the present application, the core alignment field of view is a specific field of view range used to evaluate and correct the imaging quality of the optical system during the core alignment process, and is an important parameter to ensure the optimization of the optical system performance.

[0044] Step S3: obtaining a training sample data set based on the sample image, wherein the training sample data set includes the second image.

[0045] In an embodiment of the present application, obtaining a training sample data set based on a sample image includes: obtaining a second image based on the sample image and an optical system; obtaining a training sample data set based on an error of an inherent device in the optical system and the second image. The data in the training sample data set includes the sample image, a first error, and a second error. For example, the data including the first error in the training sample data set is considered to be of the same category; the data including the second error in the training sample data set is considered to be of the same category.

[0046] Specifically, based on the construction Figure 1 After the optical system shown in the figure is installed, the actual image of the first image is collected by the image sensor in the optical system to obtain the second image. Wherein, after inputting N sample images a1, a2...an into the optical simulation software corresponding to the optical system, errors are added to the inherent components of the optical system in the optical simulation software, and the training sample data set is obtained after the N sample images are processed by the optical system. Wherein, adding errors to the inherent components of the optical system includes adding first errors b1, b2...bn to the relay lens and the lens to be aligned, and adding second errors c1, c2...cn to the alignment lens group. The first error and / or the second error can be the training sample data set obtained by processing N sample images by the optical system after adding errors.

[0047] In the embodiment of the present application, the first error includes the processing error and assembly error corresponding to the relay lens, the processing and assembly errors corresponding to the non-aligned lens group in the lens to be aligned, and the processing error corresponding to the aligned lens group in the lens to be aligned; the second error includes the assembly error corresponding to the aligned lens group in the lens to be aligned.

[0048] In an embodiment of the present application, the model training method further includes: classifying the data in the training sample data set according to the type of error of the inherent device.

[0049] like Figure 4As shown, machining and assembly errors b1, b2, ..., bn are added to the lens in the relay lens or the lens group in the lens to be aligned, and assembly errors c1, c2, ..., cn are added to the lens to be aligned that requires alignment, and a series of second images after N sample images pass through the optical system with different assembly errors are obtained as the training sample data set. For example, by combining the assembly error c1, the sample images a1, a2…an and the first errors b1, b2…bn, we obtain the data {[a1, b1, c1]… [a1,bn,c1]…[an, bn, c1]}; by combining the assembly error c2, the sample images a1, a2…an and the first errors b1, b2…bn, we obtain the data {[a1, b1, c2]… [a1, bn,c2]…[an, bn,c2]}; by combining the assembly error c2, the sample images a1, a2…an and the first errors b1, b2…bn, we obtain the data {[a1, b1, c3]… [a1, bn,c3]…[an, bn,c3]}; and by analogy, by combining the assembly error cn, the sample images a1, a2…an and the first errors b1, b2…bn, we obtain the data {[a1, b1, cn]… [a1, bn,cn]…[ an, bn,cn]}.

[0050] In some embodiments, the installation error c of the aligned lens or lens assembly itself can also be taken as a category, which includes a series of error combinations of (a1, b1) (a2, b1) ... (an, b1) (a1, b2) (a2, b2) ... (an, b2) ... (a1, bn) (a2, bn) ... (an.bn). Similarly, categories c2, c3, ... cn also include these error combinations.

[0051] Step S4, training the initial model with the sample data set until the initial model meets the preset conditions to obtain the target model.

[0052] In the embodiment of the present application, the target model outputs the misalignment corresponding to the core-adjusting lens or mirror group, that is, the assembly error, the second error. The second error is input into the model as a variable to obtain the corresponding second image (e.g., image side image), and then the second error can be inferred by inputting the second image. Other uncertain variables (first errors) are all in the model. According to the target model, the values ​​of all errors can be obtained from the second pattern.

[0053] In an embodiment of the present application, the initial model is a convolutional neural network for image regression problems. The structure of the initial model includes an input layer, a convolutional layer, a pooling layer, a fully connected layer, and an output layer. The input layer is used to receive input data (e.g., a training set) and represent the image obtained by the optical system and its associated errors in the form of an array; the convolutional layer can be a one-dimensional convolutional layer (1D Convolutional Layer), which is used to extract high-dimensional feature data from the array data; the pooling layer is used to reduce the dimension of the extracted feature values, reduce redundant information, and improve training efficiency; the fully connected layer is used to map the extracted high-dimensional feature data to a low-dimensional representation to obtain the target feature vector of the initial model; the output layer is used to output the regression target value, which corresponds to the assembly error or adjustment amount of the core-adjusting optical element.

[0054] In an embodiment of the present application, before training the initial model using the training sample data set, the focusing method further includes: dividing the training sample data set into a training set and a test set.

[0055] In an embodiment of the present application, the training sample data set can be randomly divided into a training set and a test set according to a preset ratio. The preset ratio can be 4:1 or 7:3. For example, 80% of the training sample data set is divided into a training set, and 20% of the training sample data set is divided into a test set. The training set is used to train the model, modulate the weights of the neural network in the model and optimize the model; the test set is used to verify the performance of the model and evaluate the generalization ability of the model. When dividing the data set, ensure the randomness and uniformity of the division to avoid bias in certain feature values ​​or target values ​​in the data set.

[0056] In some embodiments of the present application, before dividing the training sample data set into a training set and a test set, the core tuning method further includes: preprocessing the data in the training sample data set, and the preprocessing includes: normalization processing and standardization processing.

[0057] In some embodiments of the present application, the preset condition includes that the loss value corresponding to the preset loss function is less than the preset loss threshold, and the preset loss function includes a regression loss function (such as mean square error, MSE), and its formula is expressed as follows:

[0058] Among them, Y i represents the supervised data, Represents the predicted value of the neural network, and n represents the number of samples. The weight of the loss function is modulated to minimize the error between the predicted value and the true value. The data input is passed to the network in batches in the form of an array, and it is iteratively trained with an optimization algorithm (such as Adam or SGD) until convergence.

[0059] In some embodiments of the present application, the training process of the target model may use a supervised training method. Specifically, the training method may include: using the current batch of data in the training set to iteratively update the initial model; using the test set to determine whether the initial model updated each time meets the preset conditions, if not, using the next batch of data according to the optimization algorithm (such as the gradient descent algorithm) to perform the next update on the initial model; repeating the above steps until the initial model meets the preset conditions. The preset conditions may be that the loss value corresponding to the loss function is less than the preset loss threshold, the target model performance (such as accuracy, recall rate, etc.) reaches the preset performance threshold, the number of model iterations reaches the preset number threshold, etc., wherein the above thresholds may be set according to actual needs, and the present application does not impose specific restrictions on this.

[0060] Figure 5 is a flow chart of the core adjustment method provided in the embodiment of the present application, such as Figure 5 As shown, the core adjustment method is applied in electronic equipment. According to different requirements, the order of the steps in the flow chart can be changed, and some steps can be omitted.

[0061] Step S41, acquiring a target image.

[0062] In the embodiment of the present application, a second image corresponding to the first image is acquired by an image sensor in the optical system, and the second image is used as the target image.

[0063] Step S42, inputting the target image into the pre-trained target model to obtain the misalignment amount corresponding to the alignment mirror assembly.

[0064] In the embodiment of the present application, the target image is input to Figure 3 The target model pre-trained by the model training method shown can output the misalignment of the mirror group to be aligned. The purpose of training the input error and target image through the target model is to match the error with the image. The alignment method provided in the present application is used to output a second error corresponding to the target image regardless of the error in the system. Therefore, the target image is input to find the second error corresponding to it. When training the target model, all errors should be included as much as possible, so that the model covers all situations as much as possible, or is as close to the real situation as possible, to improve the accuracy of the model.

[0065] Step S43, after controlling the movement of the mechanical moving structure according to the misalignment amount, the target image is acquired again.

[0066] In the embodiment of the present application, the movement of the mechanical motion structure is controlled by the misalignment, and each degree of freedom of the six-degree-of-freedom displacement stage can be driven to make corresponding adjustments, thereby adjusting the corresponding position of the core-aligning lens group, and then the target image is acquired again through the image sensor. For example, the misalignment output by the target model is an x-axis offset of 2um. Since the mechanical motion structure is connected to the core-aligning lens or mirror group, the mechanical motion structure needs to be moved in the opposite direction by 2um. At this time, the position of the lens or mirror group is the correct position, and the imaging quality of the lens is improved accordingly.

[0067] Step S44, determining whether the alignment of the lens to be aligned corresponding to the alignment lens assembly is completed according to the quality of the target image acquired again.

[0068] In the embodiment of the present application, if the quality of the target image acquired again meets the requirements, it is considered that the lens to be aligned has completed the alignment; if the quality of the target image acquired again does not meet the requirements, the target image acquired again is input into the pre-trained target model to obtain the misalignment amount corresponding to the alignment lens group, and after controlling the movement of the mechanical motion structure according to the misalignment amount, the target image is acquired again until it is determined that the quality of the target image acquired again meets the requirements, and it is determined that the alignment of the lens to be aligned has been completed.

[0069] In some embodiments of the present application, the resolution of the target image acquired again can be calculated, and the calculated resolution can be compared with the preset resolution to determine whether the quality of the target image acquired again meets the requirements. When the calculated resolution is greater than or equal to the preset resolution, it is determined that the quality of the target image acquired again meets the requirements; when the calculated resolution is less than the preset resolution, it is determined that the quality of the target image acquired again does not meet the requirements. It should be noted that in other embodiments of the present application, it is also possible to determine whether the image quality meets the requirements based on the modulation transfer function (MTF), signal-to-noise ratio (SNR), dynamic range, color accuracy and distortion of the target image acquired again.

[0070] The training sample data set of the present application can be collected before the physical system is built, and the efficiency of simulation image collection is much higher than that of actual image collection, which greatly shortens the application process. After the model parameter training of the neural network is completed, inputting an image can quickly and accurately output the predicted misalignment, effectively improving the system's adjustment efficiency and accuracy. The neural network has strong universality. When facing a new optical system to be adjusted, the idea of ​​transfer learning is used to establish a small sample for training based on the previously trained neural network model parameters, and the training time is relatively short.

[0071] See also Figure 6, which is a principle block diagram of a model training device provided in an embodiment of the present application. The model training device 400 includes: an acquisition module 401, used to acquire a first image of a target object; a processing module 402, used to obtain a sample image based on the first image; the processing module 402 is also used to obtain a training sample data set based on the sample image, wherein the training sample data set includes a second image, and the second image is an image obtained after the first image passes through the optical system; the processing module 402 is also used to train an initial model through the sample data set until the initial model meets a preset condition to obtain a target model.

[0072] Another embodiment of the present application also provides an electronic device. Figure 1 The application environment is only an example. In other exemplary embodiments, the computer program product implementing the model training method and the core tuning method of the embodiment of the present application can also be run on any electronic device with sufficient computing power (such as Figure 1 In the electronic device 200 shown in the figure, various steps of the model training method and the core tuning method are executed to provide a core tuning function.

[0073] See also Figure 7 , is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 7 As shown, in one embodiment of the present application, the electronic device 200 can be a mobile phone, a tablet computer, a smart wearable device, an augmented reality (AR) / virtual reality (VR) device, a laptop computer, a netbook, etc. The embodiment of the present application does not impose any restrictions on the specific type of the electronic device 200.

[0074] like Figure 7 As shown, the electronic device 200 may include, but is not limited to, a communication module 101, a memory 102, a processor 103, an input / output (I / O) interface 104, and a bus 105. The processor 103 is coupled to the communication module 101, the memory 102, and the I / O interface 104 through the bus 105.

[0075] Those skilled in the art will appreciate that the schematic diagram is merely an example of the electronic device 200 and does not constitute a limitation on the electronic device 200 , and may include more or fewer components than shown in the diagram, or a combination of certain components, or different components. For example, the electronic device 200 may also include a network access device, etc.

[0076] The communication module 101 may include a wired communication module and / or a wireless communication module. The wired communication module may provide one or more wired communication solutions such as Universal Serial Bus (USB), Controller Area Network (CAN), etc. The wireless communication module may provide one or more wireless communication solutions such as Wireless Fidelity (Wi-Fi), Bluetooth (BT), mobile communication network, Frequency Modulation (FM), near field communication technology (NFC), infrared technology (IR), etc.

[0077] The memory 102 can be used to store computer-readable instructions and / or modules. The processor 103 implements various functions of the electronic device 200 by running or executing the computer-readable instructions and / or modules stored in the memory 102 and calling the data stored in the memory 102. The memory 102 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the electronic device 200, etc. The memory 102 can include non-volatile and volatile memories, such as: a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other storage devices.

[0078] The memory 102 may be an external memory and / or an internal memory of the electronic device 200. Furthermore, the memory 102 may be a memory in a physical form, such as a memory stick, a TF card (Trans-flash Card), and the like.

[0079] The processor 103 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. The processor 103 is the computing core and control center of the electronic device 200, and uses various interfaces and lines to connect various parts of the entire electronic device 200, and execute the operating system of the electronic device 200 and various installed applications, program codes, etc.

[0080] Exemplarily, the computer-readable instructions may be divided into one or more modules / sub-modules / units, one or more modules / sub-modules / units are stored in the memory 102 and executed by the processor 103 to complete the present application. One or more modules / sub-modules / units may be a series of computer-readable instruction segments capable of completing a specific function, and the computer-readable instruction segments are used to describe the execution process of the computer-readable instructions in the electronic device 200. For example, the computer-readable instructions may be divided into multiple modules of the above-mentioned model training device.

[0081] If the module / unit integrated in the electronic device 200 is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, and can also instruct the relevant hardware to complete it through computer-readable instructions. The computer-readable instructions can be stored in a computer-readable storage medium. When the computer-readable instructions are executed by the processor, the steps of the above-mentioned various method embodiments can be implemented.

[0082] The computer-readable instructions include computer-readable instruction codes, which may be in source code form, object code form, executable files or some intermediate form, etc. The computer-readable medium may include: any entity or device capable of carrying computer-readable instruction codes, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM).

[0083] Combination Figure 2The memory 102 in the electronic device 200 stores computer-readable instructions, and the processor 103 can execute the computer-readable instructions stored in the memory 102 to implement the following Figure 2 The alignment method shown.

[0084] Specifically, the specific implementation method of the processor 103 for the above-mentioned computer readable instructions can refer to Figure 2 The description of the relevant steps in the corresponding embodiments will not be repeated here.

[0085] The I / O interface 104 is used to provide a channel for user input or output. For example, the I / O interface 104 can be used to connect various input and output devices, such as a mouse, keyboard, touch device, display screen, etc., so that the user can enter information or visualize information.

[0086] The bus 105 is at least used to provide a channel for mutual communication among the communication module 101 , the memory 102 , the processor 103 , and the I / O interface 104 in the electronic device 200 .

[0087] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules is only a logical function division, and there may be other division methods in actual implementation.

[0088] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0089] In addition, each functional module in each embodiment of the present application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of hardware plus software functional modules.

[0090] Therefore, no matter from which point of view, the embodiments should be regarded as illustrative and non-restrictive, and the scope of the present application is limited by the appended claims rather than the above description, so it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims are included in the present application. Any attached figure mark in the claims should not be regarded as limiting the claims involved.

[0091] In addition, it is obvious that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices can also be implemented by one unit or device through software or hardware. The words first, second, etc. are used to indicate names, and do not indicate any specific order.

[0092] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present application and are not intended to limit it. Although the present application has been described in detail with reference to the preferred embodiments, a person of ordinary skill in the art should understand that the technical solution of the present application may be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present application.

Claims

1. A model training method, applied to an electronic device, wherein the electronic device is communicatively connected to an optical system, characterized in that: The method comprises: Acquire a first image of the target object; obtaining a sample image based on the first image; Obtaining a training sample data set based on the sample image, wherein the training sample data set includes a second image, and the second image is an image obtained after the first image passes through the optical system; The initial model is trained using the sample data set until the initial model meets the preset conditions, thereby obtaining a target model.

2. The model training method according to claim 1, characterized in that: The obtaining a sample image based on the first image comprises: Setting image information of the first image according to parameters of the optical system; The sample image is obtained based on the image information and the parameters of the first image.

3. The model training method according to claim 1, characterized in that: The obtaining of a training sample data set based on the sample image further comprises: obtaining the second image based on the sample image and the optical system; A training sample data set is obtained based on the error of the optical system and the second image.

4. The model training method according to claim 3, characterized in that: The error in the optical system includes a first error and a second error, wherein the first error includes a processing error corresponding to the relay lens in the optical system and / or an assembly error corresponding to the relay lens, a processing and assembly error corresponding to the non-aligned lens group in the lens to be aligned, and a processing error of the aligned lens group; the second error includes an assembly error corresponding to the aligned lens group in the lens to be aligned.

5. The model training method according to claim 3, characterized in that: The method further comprises: The data in the training sample data set are classified according to the type of the error.

6. The model training method according to claim 3, characterized in that: The image information of the first image includes image dimension, image height and image size.

7. A core alignment method, characterized in that: The method comprises: Get the target image; Input the target image to a pre-trained target model to obtain the misalignment amount corresponding to the center-aligning mirror group, wherein the target model is obtained according to any one of the model training methods described in claims 1 to 5; After controlling the movement of the mechanical motion structure according to the misalignment amount, acquiring the target image again; Whether the alignment of the lens to be aligned corresponding to the alignment lens assembly is completed is determined according to the quality of the target image acquired again.

8. A model training device, characterized in that: The model training device comprises: An acquisition module, used for acquiring a first image of a target object; A processing module, configured to obtain a sample image based on the first image; The processing module is further used to obtain a training sample data set based on the sample image, wherein the training sample data set includes a second image, and the second image is an image obtained after the first image passes through the optical system; The processing module is also used to train the initial model through the sample data set until the initial model meets the preset conditions to obtain the target model.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores at least one instruction, and when the at least one instruction is executed by the processor, it implements the model training method according to any one of claims 1 to 6 and the core tuning method according to claim 7.

10. An electronic device, characterized in that: include: A memory, and a processor, wherein the processor executes computer-readable instructions stored in the memory to implement the model training method according to any one of claims 1 to 6 and the core tuning method according to claim 7.

Images