Camera manufacturing to reduce defect rates
By using machine learning to predict and correct optical performance deviations in camera manufacturing, the method enhances production efficiency and quality by minimizing defects and optimizing assembly processes.
Patent Information
- Application Number
- JP2023555198
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Priority Date
- 2021-03-10
- Filing Date
- 2022-01-19
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2042-01-19
Smart Images

Figure 0007584675000001 
Figure 0007584675000002
Abstract
Description
[Technical field]
[0001] The present invention relates to manufacturing cameras from pre-manufactured parts. [Background technology]
[0002] The optical performance of a camera depends on the actual configuration of the ray path from the scene to be imaged to the sensor used for image capture corresponding to the pre-planning in this respect, i.e. the optical components used, such as lenses and objectives, must correspond to their respective specifications and be assembled in the correct spatial arrangement relative to one another.
[0003] In this respect, the optical performance of the final camera results from the joint action of the precision of the optical components on the one hand and the precision of the spatial arrangement of these components on the other hand. If the optical components are manufactured with very narrow tolerances, the precision of "pick-and-place" manufacturing techniques is sufficient to obtain a camera with acceptable optical performance. If cheaper optical components manufactured with wider tolerances are used, the resulting deviations of the ray paths from the preplanned can be at least partially compensated for by adjusting the components during manufacturing. In this adjustment, also called "active alignment", the components are moved relative to one another according to one or more degrees of freedom and a configuration with good optical performance is fixed by gluing the components together. DE102014220519A1 discloses an exemplary method for such an adjustment. Summary of the Invention
[0004] Within the scope of the present invention, a method for manufacturing a camera has been developed. The camera can be configured, inter alia, for monitoring the surroundings of a vehicle or at least a part of it. Such a camera can, for example, enable the driver of the vehicle to see into otherwise unobstructed areas around the vehicle, for example behind the vehicle or into "blind spots" that are not captured by the mirrors. However, driver assistance systems of the vehicle or systems for at least partially automated driving can also use camera images from the surroundings of the vehicle.
[0005] The method starts with the provision of prefabricated components, which may include, inter alia, optical components such as objectives, lenses, and image sensors, but may also include, for example, electronic circuits for further evaluation and transfer of the images captured by the image sensor, as well as a housing.
[0006] At least two of the prefabricated parts are adjusted relative to one another according to at least one predetermined optimality criterion. The parts are attached to one another in the adjusted state, i.e. when the predetermined optimality criterion is met. The attachment can be initiated, for example, by activating a photocurable adhesive with UV light.
[0007] Optimality criteria may include, for example, that the optical performance, measured based on any metric, of a combination of parts to be adjusted relative to one another is above a predetermined threshold. In addition, prior data characterizing the concrete individuality of at least one pre-manufactured part and / or measurement data on the optical performance of the combination of parts adjusted to each other are referenced. That is, only prior data, only measurement data (e.g. from an "active alignment" device), or any combination of prior data and measurement data may be used. This data is mapped by the trained machine learning model to a prediction on the optical performance that the camera will provide after at least one further manufacturing step after gluing. This prediction is referenced as feedback to influence the manufacturing process.
[0008] If measurement data on the optical performance are captured during the adjustment of the pre-manufactured components, this already provides a basis for how good the final optical performance of the camera will be. However, it has been found that the state achieved within the framework of the adjustment can be further altered by at least one further processing step that this camera carries out after the application of the components. For example, if another electronic board is mounted in a housing that already contains an optical component, this housing can be subjected to vibrations and / or heating. Furthermore, the adhesive used has not yet completely hardened immediately after application of the components. Although these components are already protected against the achieved adjustment being completely lost again due to vibrations or other handling of the camera, the adhesive can still shrink during the final hardening, which can take a few more minutes. This can once again slightly change the spatial arrangement of the applied components.
[0009] The change in the optical performance of the camera between gluing and final completion cannot be understood by a closed logic formula. However, unexpectedly, it was shown that there are correlations between the pre-data and the measurement data obtained during the adjustment on the one hand, and the final optical performance of the camera at the end of the manufacturing process on the other hand, and that these correlations do not depend on the concrete form of the further manufacturing steps after gluing. These correlations can be learned by machine learning models.
[0010] For example, if a particular component, for example a lens, is already significantly out of specification and can only be adjusted with great effort relative to the objective lens in such a way that this lens-objective lens combination allows for sharp imaging, then there is already a high probability that even a slight shrinkage of the adhesive after gluing or other changes in the camera will again move the objective lens and lens arrangement out of its very sharp optimum state, rendering the camera as a whole unusable.In contrast, if the lens and objective lens perfectly correspond to the specification, the optimum state of adjustment is significantly wider, and therefore the influence of deviations due to, for example, adhesive shrinkage on the final optical performance of the camera is significantly weaker.
[0011] To train the machine learning model, for example, the prior data and measurement data on the one hand and the optical performance measured after the completion of the camera on the other hand for a certain number of training cameras can be used as training data. In this case, for example, the parameters characterizing the behavior of the machine learning model can be optimized with the goal that the machine learning model correctly maps the respective prior data and measurement data for each of the training cameras to the respective optical performance after completion. If the training cameras and the attributive prior data and measurement data have sufficient diversity, it can be expected that the machine learning model's great power for generalization will also correctly predict the optical performance of a camera that has never been observed before and will be produced for the first time later.
[0012] The machine learning model to be trained may include and / or be, inter alia, a neural network, for example. During training of the neural network, inter alia, weights as parameters are optimized. Using these weights, inputs sent to multiple neurons and / or other processing units are combined by forming a weighted sum to activate one neuron or processing unit. From this activation, the output of this one neuron or processing unit is determined by applying a predefined nonlinearity, for example a ReLU function or a sigmoid function.
[0013] The ultimate use of this prediction is that a possibly predictable poor optical performance of the camera can be recognized as early as possible in the manufacturing process, ideally so that the predictable poor performance can be precisely improved by the application of appropriate measures in the manufacturing process, or if this is not possible, then the camera that cannot be "saved" can at least be discarded early as a defective product, before any further refining processing steps are performed.
[0014] If, for example, during adjustments, it has already been determined that the finished camera will not function properly, there is no point in adding another electronic board and gluing it together with the optics. Any further time spent on this camera from the point at which it cannot be "saved" is a waste. In this case, the electronic board itself may also be wasted, since manually extracting it from the abandoned camera in order to install it in a new camera may cost more than the board itself, if that is technically possible at all.
[0015] The situation is somewhat similar to university study: students are under particular pressure to perform well in foundational subjects at the beginning of their studies and receive feedback on any weak areas they may have. Ideally, this feedback can be used to precisely address these weaknesses and complete their studies successfully. If this is not possible, the student can be assessed early enough to move on to further study or vocational training.
[0016] In a particularly advantageous embodiment, a number of candidate pieces are prepared for at least one type of component. With reference to prior data characterizing each of these candidate pieces, a machine learning model determines a prediction for the optical performance of a camera in which the candidate pieces are included. For the subsequent manufacture of the camera, a combination of the components of the component whose prediction meets a predetermined criterion is selected.
[0017] This results in more functioning cameras overall when the individual characteristics of the parts vary relatively strongly based on prior manufacturing. That is, for example, in the case of a ray path based on optical components connected in series, the deviations of these components from the standard can mutually strengthen or mutually weaken the effect of these components on the final optical performance of the camera, depending on the direction in which each component deviates from the standard. If there is a pool of individual components for each component, then it is possible to use predictions based on machine learning models to precisely combine individual components from the respective pools, which mutually weaken the deviations from the standard. In this way, a camera that is still capable of functioning can be produced from components that are themselves of poor quality. In this case, it is not necessarily necessary to draw definite boundaries for the deviations from the standard of the individual components and to discard components that exceed this boundary as defective.
[0018] In a further advantageous embodiment, measurement data is collected several times during the adjustment regarding the optical performance of the combination of parts in the respective spatial arrangement relative to each other. From these measurement data, a machine learning model determines a prediction regarding the optical performance of the camera that will result when the parts are attached to each other in the respective arrangement relative to each other. In response to the prediction satisfying a predetermined criterion, the parts are attached to each other. In this way, for example, further deterioration caused by subsequent manufacturing steps can be predicted and at least partially compensated for before attachment. This improves the final optical performance of the camera after the manufacturing is completed. This is somewhat similar to how fruit that is shipped to a customer over a sea route that lasts several weeks is not yet fully ripe when it is sent on the ship, so that it arrives at the customer before it spoils.
[0019] Thus, inter alia, during tuning, preference may be given to an optimization taking into account predictions provided by a machine learning model over an optimization taking into account a predefined optimality criterion. As explained above, in response to predictions regarding the optical performance of the camera meeting a predetermined criterion, the manufacturing process may be interrupted. In this way, the use of additional time and materials for cameras that can no longer be "salvaged" may be avoided. Overall, a given quantity of functioning cameras may thus be manufactured with less material consumption.
[0020] Prior data may, inter alia, be The modulation transfer function (MTF) of the optical components and / or Measurement results from quality inspection of components within the framework of pre-manufacturing and / or The source of the part and / or at least one tool used in the manufacture of the part may be characterized as:
[0021] The modulation transfer function can be determined, for example, by imaging an object with at least one sharp edge by the optical component, which image is ideally also sharp, and the MTF can then be quantified by determining a characteristic value for the actual sharpness of the image.
[0022] Measurements from other quality inspections may include, for example, the angle of unwanted field tilt caused by optics. The source of the parts can be important, for example, in that the manufacturing processes, and therefore also the tolerances, are differentiated from one source to another. Also, for example, the tools used by one and the same source, for example injection molds for injection molded parts, can have small deviations from each other. These deviations can direct the deviations of the parts produced by these tools, respectively, from the standard in certain directions.
[0023] In a further advantageous embodiment, the measurement data characterizes the modulation transfer function MTF of the combination of parts adjusted relative to each other and / or the dimensions of the spatial arrangement of parts adjusted relative to each other. Just as the optical performance of the finished camera is measured with MTF, the pre-measurement of MTF during adjustment provides a good basis for the performance of the finished camera. Through the spatial arrangement of the parts during adjustment, the deviation of the parts from the standard can be approximately quantified. For example, if a certain part has to be moved particularly far from its expected position to adjust to another part, this is evidence that one of these parts is out of standard.
[0024] The method may, inter alia, be computer-implemented, i.e., embodied in, for example, a computer program having machine-readable instructions that, when executed by one or more computers controlling a camera manufacturing facility, cause the manufacturing facility to perform the aforementioned method.
[0025] The invention likewise relates to a machine-readable data carrier comprising a computer program and / or a download product, which is a digital product that can be transmitted over a data network, i.e. downloaded by a user of the data network, and which can for example be offered for sale in an online shop for immediate download.
[0026] Additionally, the computer may be equipped with a computer program, a machine-readable data storage medium, or a downloadable product. The invention further relates to a computer program, a machine-readable data storage medium, a download product, and / or a camera manufacturing facility comprising one or more computers.
[0027] In the following, further measures for improving the invention are presented in more detail together with a description of preferred exemplary embodiments of the invention based on the figures. [Brief description of the drawings]
[0028] [Figure 1] 1 illustrates an exemplary embodiment of a method 100 for manufacturing a camera 1. FIG. [Diagram 2] FIG. 2 shows an exemplary advantageous pairing of individual pieces 2a1-2a5, 2b1-2b5 of parts 2a, 2b, each manufactured with tolerances. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0029] FIG. 1 is a schematic flow diagram of one exemplary embodiment of a method 100 for manufacturing at least one camera 1 . In step 110, pre-manufactured parts 2a, 2b are provided. In this regard, for example, based on block 111, a plurality of candidate pieces 2a1-2a5 or 2b1-2b5 may be provided for each of the parts 2a, 2b, respectively.
[0030] In step 120, at least two pre-manufactured parts 2a, 2b are adjusted relative to each other according to at least one predetermined optimality criterion. In step 130, these parts 2a, 2b are glued together in the adjusted state.
[0031] In step 140, a priori data 4a, 4b characterizing concrete individuals 2a1-2a5, 2b1-2b5 of at least one type of pre-manufactured part 2a, 2b and / or Measurement data 5 relating to the optical performance of the combination of the components 2 a, 2 b adjusted relative to one another, The trained machine learning model 6 maps to a prediction 7 regarding the optical performance that the camera 1 will provide after at least one further manufacturing step has been performed after gluing. In step 150, this prediction 7 is referenced as feedback in order to determine an influence 8 on the manufacturing process, which may be on the preparation 110 of the parts 2a, 2b, on the adjustment 120 of the parts 2a, 2b to one another and / or on the gluing 130 of the parts.
[0032] In particular, for example, based on block 141, a prediction 7 regarding the optical performance of the camera 1 in which each of the candidate individuals 2a1-2a5, 2b1-2b5 is contained can be determined by the machine learning model 6 with reference to the prior data 4a, 4b characterizing each of the candidate individuals 2a1-2a5, 2b1-2b5, respectively. Then, based on block 151, one combination of the individuals 2a1-2a5, 2b1-2b5 of the parts 2a, 2b can be selected for the subsequent manufacture of the camera 1, such that this prediction 7 meets a predetermined criterion. The criterion can include, for example, that the prediction 7 is maximized or at least above a predetermined threshold. The selection of the combination of the individuals 2a1-2a5, 2b1-2b5 is one impact 8 on the preparation 110 of the parts 2a, 2b.
[0033] Based on block 142, for example several times during the adjustment, measurement data 5 can be captured regarding the optical performance of the combination of parts 2a, 2b in the respective spatial arrangement relative to one another, i.e. in the respective state of adjustment. From these measurement data 5, a prediction 7 can then be determined by a machine learning model 6 regarding the optical performance of the camera 1 that would result if the parts 2a, 2b were glued to one another in this arrangement, i.e. in the case where the respective state of adjustment is fixed. Then, in block 152, it can be checked whether this prediction satisfies a predefined criterion, i.e. for example is maximal or at least exceeds a predefined threshold value. If this is the case (true / false value 1), then based on block 155, the parts 2a, 2b can be glued to one another. The start of the gluing 130 depending on the prediction 7 is one effect 8 on the gluing 130.
[0034] Also based on block 121, during the adjustment 120, priority may be given to an optimization taking into account a prediction 7 provided by a machine learning model 6 over an optimization taking into account a predefined optimality criterion, that is to say, the prediction 7 again has an influence 8 on the adjustment 120.
[0035] Furthermore, according to block 153, it can be checked whether the prediction 7 regarding the optical performance of the camera meets a predefined criterion, i.e. for example lies below a predefined threshold. If this is the case (true / false value 1), according to block 154, the manufacturing process can be interrupted.
[0036] FIG. 2 illustrates, by way of example, how predictions 7 determined by a machine learning model 6 can be used to increase the yield of functioning cameras 1 and reduce rejects when manufacturing cameras 1 from parts 2a, 2b whose pre-manufactured characteristics have a certain degree of tolerance.
[0037] In the example shown in FIG. 2, the component 2a is a lens, of which five different components 2a1-2a5 are available to choose from. The component 2b is an objective lens, of which five different components 2b1-2b5 are available to choose from. The components 2a1-2a5, 2b1-2b5 are each characterized by a number indicating an exemplary deviation of the optical parameters of the respective components 2a1-2a5, 2b1-2b5 from the standard for the respective components 2a, 2b. For illustrative purposes, it is assumed that the respective deviations from the standard are additive when the lens 2a and the objective lens 2b are connected in tandem.
[0038] In the example shown in Figure 2, based on prediction 7, combination Lens 2a1 and objective lens 2b5, Lens 2a2 and objective lens 2b1, Lens 2a3 and objective lens 2b2, and Lens 2a5 and objective lens 2b3, It can be established that in this case the respective deviations are nullified exactly. The combination of lens 2a4 and objective lens 2b4 produces only a small deviation which does not have to be corrected afterwards.
Claims
1. - a step (110) in which pre-manufactured parts (2a, 2b) are provided; - a step (120) in which at least two of said pre-manufactured parts (2a, 2b) are adjusted relative to one another according to at least one predetermined optimality criterion; A step (130) in which the parts (2a, 2b) are attached to each other in an adjusted state; A method (100) for manufacturing a camera (1), comprising: Additionally, a step (140) in which prior data (4a, 4b) characterizing concrete individuals (2a1-2a5, 2b1-2b5) of at least one of said pre-manufactured parts (2a, 2b) and / or measurement data (5) on the optical performance of a combination of said parts (2a, 2b) adjusted to one another is mapped by a trained machine learning model (6) to a prediction (7) on the optical performance that said camera (1) will provide after at least one further manufacturing step after gluing; and a step (150) in which the prediction (7) is referred to as feedback to influence (8) the manufacturing process, A plurality of candidate objects (2a1 to 2a5, 2b1 to 2b5) are prepared for at least one type of part (2a, 2b) (111); With reference to prior data (4a, 4b) characterizing each of the candidate objects (2a1-2a5, 2b1-2b5), the machine learning model (6) determines (141) a prediction (7) regarding the optical performance of a camera (1) in which the candidate objects (2a1-2a5, 2b1-2b5) are contained, and The method (100) selects (151) one combination of the individuals (2a1-2a5, 2b1-2b5) of the parts (2a, 2b) for which the prediction (7) meets a predetermined criterion for subsequent manufacturing of the camera (1).
2. A step (110) in which pre-manufactured parts (2a, 2b) are provided; - a step (120) in which at least two of said pre-manufactured parts (2a, 2b) are adjusted relative to one another according to at least one predetermined optimality criterion; A step (130) in which the parts (2a, 2b) are attached to each other in an adjusted state; A method (100) for manufacturing a camera (1), comprising: Additionally, a step (140) in which prior data (4a, 4b) characterizing concrete individuals (2a1-2a5, 2b1-2b5) of at least one of said pre-manufactured parts (2a, 2b) and / or measurement data (5) on the optical performance of a combination of said parts (2a, 2b) adjusted to one another is mapped by a trained machine learning model (6) to a prediction (7) on the optical performance that said camera (1) will provide after at least one further manufacturing step after gluing; and a step (150) in which the prediction (7) is referred to as feedback to influence (8) the manufacturing process, From measurement data (5) relating to the optical performance of the combination of the parts (2 a, 2 b) in each spatial arrangement relative to one another at multiple times during the adjustment, a prediction (7) is determined (142) by the machine learning model (6) regarding the optical performance of the camera (1) that will result when the parts (2 a, 2 b) are attached to one another in said arrangement; and In response to the prediction (7) satisfying (152) a predetermined criterion, the parts (2a, 2b) are attached (130, 155) to one another, the method (100).
3. A step (110) in which pre-manufactured parts (2a, 2b) are provided; - a step (120) in which at least two of said pre-manufactured parts (2a, 2b) are adjusted relative to one another according to at least one predetermined optimality criterion; A step (130) in which the parts (2a, 2b) are attached to each other in an adjusted state; A method (100) for manufacturing a camera (1), comprising: Additionally, a step (140) in which prior data (4a, 4b) characterizing concrete individuals (2a1-2a5, 2b1-2b5) of at least one of said pre-manufactured parts (2a, 2b) and / or measurement data (5) on the optical performance of a combination of said parts (2a, 2b) adjusted to one another is mapped by a trained machine learning model (6) to a prediction (7) on the optical performance that said camera (1) will provide after at least one further manufacturing step after gluing; and a step (150) in which the prediction (7) is referred to as feedback to influence (8) the manufacturing process, The method (100) interrupts (154) the manufacturing process in response to the prediction (7) regarding the optical performance of the camera (1) meeting (153) a predetermined criterion.
4. A step (110) in which pre-manufactured parts (2a, 2b) are provided; - a step (120) in which at least two of said pre-manufactured parts (2a, 2b) are adjusted relative to one another according to at least one predetermined optimality criterion; A step (130) in which the parts (2a, 2b) are attached to each other in an adjusted state; A method (100) for manufacturing a camera (1), comprising: Additionally, a step (140) in which prior data (4a, 4b) characterizing concrete individuals (2a1-2a5, 2b1-2b5) of at least one of said pre-manufactured parts (2a, 2b) and / or measurement data (5) on the optical performance of a combination of said parts (2a, 2b) adjusted to one another is mapped by a trained machine learning model (6) to a prediction (7) on the optical performance that said camera (1) will provide after at least one further manufacturing step after gluing; and a step (150) in which the prediction (7) is referred to as feedback to influence (8) the manufacturing process, The prior data is the modulation transfer function MTF of the optical components (2a, 2b) and / or Measurement results from a quality inspection of the components (2a, 2b) within the framework of prior manufacture and / or The source of the parts (2a, 2b) and / or at least one tool used in the manufacture of the parts (2a, 2b). The method (100) characterizes
5. The method (100) according to claim 2, wherein during the tuning (120), priority is given (121) to an optimization taking into account the predictions (7) provided by the machine learning model (6) over an optimization taking into account the predetermined optimality criterion.
6. The measurement data (5) the modulation transfer function MTF of the combination of said components (2a, 2b) adjusted relative to one another and / or the dimensions of the spatial arrangement of said parts (2a, 2b) adjusted relative to each other The method (100) according to any one of claims 1 to 5, characterized in that
7. The method (100) of any one of claims 1 to 6, wherein the prediction (7) of optical performance characterizes a modulation transfer function MTF of the completed camera (1).
8. 8. A computer program comprising machine-readable instructions which, when executed on one or more computers, cause the one or more computers to carry out a method according to any one of claims 1 to 7 in connection with a camera manufacturing facility controlled by the one or more computers.
9. A machine-readable data storage medium and / or download product comprising a computer program according to claim 8.
10. One or more computers comprising a computer program according to claim 8 and / or a machine-readable data storage medium and / or a download product according to claim 9.
11. A camera manufacturing facility equipped with a computer program according to claim 8, a machine-readable data storage medium and / or a download product according to claim 9, and / or one or more computers according to claim 10.
Citation Information
Patent Citations
Method for positioning an image - capturing element relative to an optical device and use of the method
DE102014220519A1
Eccentricity adjustment method, apparatus, and program for lens optical system
JP2008170981A
Manufacturing system, manufacturing method for assembly, and program
JP2018151575A
Optical component position adjustment support device, method for supporting optical component position adjustment, optical component position adjustment support program, and method for manufacturing lens device
WO2020158325A1
