A visual-tactile sensor and a light source design method thereof and related equipment

Abstract

The application discloses a visual tactile sensor, a light source design method thereof and related equipment. The visual tactile sensor contains a physical light source, the physical light source includes a plurality of light emitting units of a plurality of logical lighting channels, and the method includes the following steps: obtaining a sensor mechanical model, extracting a feasible layout area and a light transmission model from the sensor mechanical model, determining a global objective function by combining a physical light source layout of the feasible layout area, a driving weight parameter and the light transmission model, and solving the global objective function according to a preset evaluation index to determine a layout coordinate of the physical light source and an optimal driving weight. The evaluation index includes condition number minimization and / or uniformity maximization. The application can optimize the illumination light field and improve the detection accuracy of the visual tactile sensor. The application can be widely applied to the technical field of visual tactile sensors.

Description

Technical Field

[0001] This application relates to the field of visual-tactile sensor technology, and in particular to a visual-tactile sensor, its light source design method, and related equipment. Background Technology

[0002] Visor-tactile sensors capture the deformation of an elastomer sensing layer using optical imaging devices, thereby calculating the three-dimensional morphology and mechanical information of the contact surface. Among them, photometric stereo, a shape-from-shading-based 3D reconstruction technique, has become the mainstream algorithm framework for high-precision visor-tactile perception due to its ability to reconstruct the microscopic texture and high-frequency geometric features of object surfaces at high resolution. The core idea of ​​the photometric stereo method is based on the theory of light irradiance. By observing the brightness changes of the same surface under different illumination directions, the normal vector field of the surface is solved, and then the three-dimensional depth information is reconstructed.

[0003] In practical products, cost considerations typically lead to the use of three-directional lighting (e.g., Gelsight). However, due to the limited size of actual products, ideal lighting conditions cannot be achieved, resulting in a deterioration of the condition number and reduced accuracy in reconstructing the height of the 3D surface. Therefore, these technical problems urgently need to be addressed. Summary of the Invention

[0004] The main purpose of this application is to propose a visual-tactile sensor, its light source design method, and related equipment, aiming to improve the detection accuracy of the visual-tactile sensor.

[0005] To achieve the above objectives, one aspect of this application proposes a light source design method for a visual-tactile sensor, wherein the visual-tactile sensor includes a physical light source, and the physical light source includes several light-emitting units with several logical illumination channels. The method includes the following steps: Obtain the sensor mechanical model, and extract the feasible layout region and optical transmission model from the sensor mechanical model; The global objective function is determined by combining the physical light source layout and driving weight parameters of the feasible layout region with the optical transmission model. The global objective function is solved according to the preset evaluation index to determine the layout coordinates of the physical light source and the optimal driving weight; the evaluation index includes minimizing the condition number and / or maximizing uniformity.

[0006] In this invention, the determination of the global objective function by combining the physical light source layout and driving weight parameters of the feasible layout region with the optical transmission model includes: The transmission function is determined based on the light transmission model from the light-emitting unit at a specified position in the feasible layout area to the specified target point on the sensing surface of the visual-touch sensor in the preset logical lighting channel. The synthetic light vector of the specified target point is determined based on the transfer function and driving weight parameters; A global objective function is formed based on the synthesized light vectors of multiple specified target points on the sensing surface.

[0007] In this invention, the step of solving the global objective function according to a preset evaluation index to determine the layout coordinates of the physical light source and the optimal driving weight includes: A region of interest is determined on the sensing surface of the visual-touch sensor, and a global average conditional function for all target points within the region of interest is determined based on the global objective function. Minimize the global average condition function to determine the layout coordinates of the physical light source and the optimal driving weight.

[0008] In this invention, the step of solving the global objective function according to a preset evaluation index to determine the layout coordinates of the physical light source and the optimal driving weight includes: The ratio function of the maximum and minimum composite light vectors among all target points on the sensing surface is determined based on the global objective function. With the preset evaluation index being the maximization of uniformity, the layout coordinates of the physical light source and the optimal driving weight are determined by minimizing the proportional function.

[0009] In this invention, the step of solving the global objective function according to a preset evaluation index to determine the layout coordinates of the physical light source and the optimal driving weight includes: A region of interest is determined on the sensing surface of the visual-touch sensor, and a global average conditional function for all target points within the region of interest is determined based on the global objective function. The vector ratio function between the maximum and minimum composite light vectors among all target points on the sensing surface is determined based on the global objective function. The global cost function is determined based on the global average condition function, vector scale function, and preset weights. The global cost function is minimized to determine the layout coordinates of the physical light source and the optimal driving weights.

[0010] To achieve the above objectives, another aspect of this application proposes a visual-tactile sensor, including a physical light source, an optical element, a controller, and a sensing surface. Light emitted from the physical light source passes through the optical element and reaches the sensing surface. A plurality of light-emitting units of the physical light source are installed according to the above-described layout coordinates. The controller is used to control the luminous intensity of the plurality of light-emitting units of the physical light source according to the above-described optimal driving weights. The optical element is determined according to the above-described light transmission model.

[0011] In this invention, the modulation method of the optimal driving weight includes pulse width modulation or amplitude modulation.

[0012] To achieve the above objectives, another aspect of this application proposes a light source design device for a visual-tactile sensor, wherein the visual-tactile sensor includes a physical light source, the physical light source comprising a plurality of light-emitting units with a plurality of logical illumination channels, and the device includes: The first module is used to acquire a sensor mechanical model and extract a feasible layout region and an optical transmission model from the sensor mechanical model. The second module is used to determine the global objective function by combining the physical light source layout and driving weight parameters of the feasible layout area with the optical transmission model. The third module is used to solve the global objective function according to the preset evaluation index, and determine the layout coordinates of the physical light source and the optimal driving weight; the evaluation index includes minimizing the condition number and / or maximizing uniformity.

[0013] To achieve the above objectives, another aspect of this application proposes an electronic device comprising a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the methods described above.

[0014] To achieve the above objectives, another aspect of this application proposes a computer-readable storage medium storing a computer program that, when executed by a processor, implements the methods described above.

[0015] To achieve the above objectives, another aspect of this application proposes a computer program product, including a computer program that, when executed by a processor, implements the methods described above.

[0016] This application includes at least the following beneficial effects: This application provides a visual-tactile sensor and its light source design method and related equipment. The scheme extracts feasible layout regions and light transmission models through sensor mechanical models, determines the global objective function by combining the physical light source layout and driving weight parameters of the feasible layout regions with the light transmission model, solves the global objective function according to preset evaluation indicators, determines the layout coordinates of the physical light source and the optimal driving weight, and the evaluation indicators include minimizing the condition number and / or maximizing uniformity. By optimizing the spatial layout of the physical light source and the driving weight parameters in a coordinated manner, the consistency and signal-to-noise ratio of the illumination light field are improved, thereby optimizing the illumination light field and improving the reconstruction accuracy. Attached Figure Description

[0017] Figure 1 This is a flowchart of the light source design method in the visual-tactile sensor provided in the embodiments of this application; Figure 2 This is a flowchart of the light source design method and application method in the visual-tactile sensor provided in the embodiments of this application; Figure 3 This is a schematic diagram of the light-emitting unit layout of the ring sensor provided in the embodiments of this application; Figure 4 This is a condition number distribution diagram of the sensing surface of the ring sensor provided in the embodiments of this application; Figure 5 This is a schematic diagram of the light-emitting unit layout of the rectangular ring sensor provided in an embodiment of this application; Figure 6 This is a condition number distribution diagram of the sensing surface of the rectangular ring sensor provided in the embodiments of this application; Figure 7 This is a schematic diagram of the illumination vector synthesis principle of an illumination logic channel composed of multiple physical light-emitting units, provided in an embodiment of this application. Figure 8 This is a schematic diagram of the light source design device in the visual-tactile sensor provided in the embodiments of this application; Figure 9 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit it. In the following description, when referring to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with those of this application; they are merely examples of apparatuses and methods consistent with some aspects of the embodiments of this application as detailed in the appended claims.

[0019] It is understood that the terms “first,” “second,” etc., used in this application may be used herein to describe various concepts, but unless otherwise stated, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another. For example, without departing from the scope of the embodiments of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the words “if,” “when,” or “in response to a determination” as used herein may be interpreted as “when…” or “when…” or “in response to a determination.”

[0020] As used in this application, the terms "at least one", "multiple", "each", "any", etc., "at least one" includes one, two or more, "multiple" includes two or more, "each" refers to each of the corresponding multiples, and "any" refers to any one of the multiples.

[0021] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0022] In typical configurations of visual-tactile sensors, it is generally assumed that the coating on the elastomer surface has ideal Lambertian reflectivity. According to Lambertian cosine law, the radiance of a pixel in an image... i With the incident light vector l and surface normal vector N The relationship between them can be represented as:

[0023] Where ρ is the surface albedo. When using three or more linearly independent light source directions... When illuminating the same scene, a system of linear equations can be constructed:

[0024] Ideally, three sets of linearly independent light source vectors are needed to calculate the surface normal vector at that point:

[0025] Subsequently, by solving the Poisson equation or using path integration on the normal vector field (gradient field), the three-dimensional height map of the contact surface is finally reconstructed.

[0026] Due to system noise such as shot noise, dark noise, and circuit noise, the image signal captured by the camera fluctuates, which affects the stability of the normal vector calculation. The characteristics of the light field L determine the degree of influence of gray-level noise on the normal vector calculation. According to linear algebra theory, the condition number of the illumination matrix L can quantify the amplification of camera gray-level noise transmitted to the normal vector calculation noise. An ideal linearly independent illumination environment is one where the illumination vectors of the three light sources at that point are mutually orthogonal and have consistent intensity, and the condition number of the illumination matrix L at that point is 1. The smaller the condition number of the illumination light field, the less the stability of the normal vector calculation is affected by pixel gray-level noise, and the higher the calculation accuracy.

[0027] The light source design method for a visual-tactile sensor provided in this application relates to the field of visual-tactile sensor technology. This method can be applied to a terminal, a server, or software running on a terminal or server. In some embodiments, the terminal can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, or in-vehicle terminal, but is not limited thereto. The server can be configured as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The server can also be a node server in a blockchain network. The software can be an application implementing the light source design method for a visual-tactile sensor, but is not limited to the above forms.

[0028] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics devices, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0029] Figure 1 This is an optional flowchart of a light source design method for a visual-tactile sensor provided in this application embodiment. The visual-tactile sensor includes a physical light source, which comprises several light-emitting units with several logical illumination channels. Figure 1 The method may include, but is not limited to, steps S101 to S103.

[0030] Step S101: Obtain the sensor mechanical model, and extract the feasible layout region and optical transmission model from the sensor mechanical model; Step S102: Determine the global objective function by combining the physical light source layout and driving weight parameters of the feasible layout region with the optical transmission model; Step S103: Solve the global objective function according to the preset evaluation index to determine the layout coordinates of the physical light source and the optimal driving weight; the evaluation index includes minimizing the condition number and / or maximizing uniformity.

[0031] This invention employs an equivalent light field synthesis technique based on the principle of light vector superposition. Addressing the issues of uneven illumination and difficulty in controlling the illumination direction caused by the limited internal space of visual-tactile sensors and the extremely close proximity of the light source to the sensing surface (near-field illumination), this invention achieves high-precision photometric stereo perception by constructing several (e.g., K) logical illumination channels. A "logical illumination channel" refers to an independent illumination component with a specific spatial orientation and ideal distribution characteristics required at the algorithmic solution level.

[0032] See Figure 2 The light source in visual-tactile sensors includes both the design and application phases.

[0033] Phase 1: Offline simulation and parameter optimization.

[0034] Constraint Modeling: Import the sensor's mechanical model, which is the overall mechanical structure, including all components of the vision sensor, such as the mechanical housing, optical elements, and circuit boards. Extract the feasible layout area of ​​the physical light source. And the optical transmission model. The global objective function is determined by combining the physical light source layout and driving weight parameters of the feasible layout region with the optical transmission model. , P The coordinates representing the physical layout of the light source. W This represents the driving weight.

[0035] Joint solution: A nonlinear optimization algorithm (such as a genetic algorithm nested with sequential quadratic programming) is used to solve the above objective function. .

[0036] Parameter fixation: Outputs the globally optimal physical coordinate distribution P Used in PCB manufacturing, outputting the optimal weight matrix. W Used for firmware burning.

[0037] Phase Two: Online Operation and Light Field Reconstruction.

[0038] Logical channel mapping: The controller maps K logical channels to K physical acquisition states according to the specific signal multiplexing protocol (such as time division multiplexing, spectral multiplexing, frequency division multiplexing, etc.).

[0039] Vector synthesis driven: In the k-th acquisition state, the controller reads the weight vector w. The controller simultaneously drives the physical light source array, where the output intensity of the i-th light source is determined by... Strict control is applied. At this point, the k-th equivalent light vector field, after mathematical correction, is reproduced in physical space.

[0040] 3D reconstruction: The image sensor acquires images and separates the photometric maps corresponding to K logical channels. The surface normal vectors and depth information are calculated using the standard photometric stereo algorithm.

[0041] In some embodiments, the global objective function is determined by combining the physical light source layout and driving weight parameters of the feasible layout region with the optical transmission model, including: Step S201: Determine the transmission function from the light-emitting unit at a specified position in the feasible layout area to the specified target point on the sensing surface of the visual-touch sensor in the preset logical lighting channel according to the light transmission model. Step S202: Determine the synthetic light vector of the specified target point based on the transfer function and driving weight parameters; Step S203: A global objective function is formed based on the composite light vector of multiple specified target points on the sensing surface.

[0042] Based on the principle of linear vector superposition of incoherent light, the synthesized light vector at any infinitesimal element q on the sensing surface is... This can be expressed as the light vector generated at the infinitesimal element q by the m-th physical light-emitting unit in the k-th logical illumination channel within the sensor cavity. Vector weighted sum:

[0043]

[0044] in, Light source Transfer function to target point q The function outputs a vector containing both intensity and direction. Let m be the spatial coordinates of the m-th physical light-emitting unit in the k-th logical channel. This contributes to the driving weight of the m-th physical light-emitting unit in the k-th logic channel. M represents the total number of light sources in the k-th group. This invention achieves precise control over the spatial distribution of physical light sources. P and driving weights W In mathematics, multiple non-ideal physical light field vectors are combined to form a logical light field with near-ideal characteristics.

[0045] In some embodiments, the global objective function is solved according to a preset evaluation index to determine the layout coordinates of the physical light source and the optimal driving weights, including: Step S301: Determine the region of interest on the sensing surface of the visual-touch sensor, and determine the global average condition function of all target points within the region of interest based on the global objective function; Step S302: Minimize the global average condition function to determine the layout coordinates of the physical light source and the optimal driving weight.

[0046] System condition number average minimization term This project aims to optimize the orthogonality of the logical light field and improve the numerical stability of the 3D reconstruction algorithm across the entire ROI (Region of Interest).

[0047] This invention uses the global average condition number as an evaluation index:

[0048] in: The area of ​​the region of interest; Let be the condition number of the illumination field matrix on the q-element.

[0049] Physical meaning: By minimizing this integral term, the three logical lighting directions are forced to... Keep as orthogonal as possible to any point within it. To avoid the occurrence of "ill-conditioned solution regions", we can approach 1), thereby ensuring the global consistency accuracy of normal vector calculation.

[0050] In some embodiments, the global objective function is solved according to a preset evaluation index to determine the layout coordinates of the physical light source and the optimal driving weights, including: Step S401: Determine the ratio function of the maximum and minimum composite light vectors among all target points on the sensing surface based on the global objective function; Step S402: With the preset evaluation index being the maximization of uniformity, the layout coordinates of the physical light source and the optimal driving weight are determined by minimizing the proportional function as the solution objective.

[0051] Illumination uniformity optimization item This measure aims to avoid local low-light areas and ensure the global signal-to-noise ratio (SNR).

[0052]

[0053] Suppressing "hot spots" and "dark areas": The focus is on eliminating overly bright centers due to near-field effects and underly dark edges due to distance attenuation. Ensuring a lower signal-to-noise ratio (SNR): In photometric stereo imaging, signals from low-light areas (dark areas) are often overwhelmed by sensor noise, leading to reconstruction failure in those areas. This is achieved by minimizing... This ensures that the light intensity in all areas within the ROI remains within the camera's optimal linear response range, thereby achieving a uniform and sufficient signal-to-noise ratio.

[0054] In some embodiments, the global objective function is solved according to a preset evaluation index to determine the layout coordinates of the physical light source and the optimal driving weights, including: Step S501: Determine the region of interest on the sensing surface of the visual-touch sensor, and determine the global average condition function of all target points within the region of interest based on the global objective function; Step S502: Determine the vector ratio function of the maximum and minimum composite light vectors among all target points on the sensing surface based on the global objective function; Step S503: Determine the global cost function based on the global average condition function, vector scale function, and preset weights, minimize the global cost function, and determine the layout coordinates of the physical light source and the optimal driving weights.

[0055] First, a Region of Interest (ROI) is defined on the sensing surface of the visual-touch sensor, denoted as . . It covers the effective working range of tactile perception, eliminating invalid areas such as mechanically obstructed areas or areas with severe optical distortion at the sensor edges. All subsequent optimization calculations are strictly limited to... Within the scope.

[0056] Construct the following global cost function for the physical light source layout. P and driving weights W Collaborative optimization:

[0057] in, This represents the global average conditional function. This represents a vector scaling function. The preset weights are determined based on the actual application; this embodiment does not impose specific limitations.

[0058] The present application will be described below with reference to several specific embodiments.

[0059] Example 1: Joint optimization of light field of visual-tactile sensor based on confined annular region.

[0060] This embodiment provides a method for designing an illumination field under the constraint of a circular ring in physical space.

[0061] 1. Initial settings for physical space and light-emitting units In this embodiment, K=3 logical lighting channels are constructed (corresponding to R, G, and B channels respectively). The distributed physical light source array at the execution end uses RGB LEDs, and the number of physical light-emitting units corresponding to each logical channel is M1=6, M2=6, and M3=5 respectively.

[0062] Feasible layout area D feas Spatial coordinates of the physical light-emitting unit It is strictly confined to an annular area with an inner radius of 10mm and an outer radius of 20mm. Within this area, interference from components and limitations on PCB routing are eliminated, allowing for random, discrete placement of the physical light source.

[0063] Region of Interest (ROI) The detection surface (such as the surface of an elastomer) is set at a vertical height of 8mm from the light source surface, and the effective detection area is a square area with a side length of 10mm.

[0064] 2. Construction of the Space-Intensity Joint Optimization Model In order to transform the constrained physical light field into an ideal logical light field, the system establishes a global objective function J(P,W) with the physical light source position P and the driving weight W as coupling variables.

[0065] exist The region is discretized into multiple infinitesimal elements q. For the k-th logic channel, its synthesized light vector at infinitesimal element q is... The light vector generated by all physical light sources within this channel The weighted sum, i.e. .

[0066] Condition number minimization term Extract the illumination matrix L, which is composed of the composite light vectors of the three logical channels at the infinitesimal element q, and calculate its condition number. Through optimization This forces the three logical lighting directions to be as orthogonal as possible across the entire domain, i.e. Close to 1.

[0067] Uniformity maximization term Extract each logical channel in The maximum and minimum illuminance amplitude within the range are penalized. It suppresses the central hot spot and edge dark area of ​​near-field illumination.

[0068] 3. Offline simulation and joint solution Based on the extracted mechanical constraints and optical transmission model, a nonlinear optimization algorithm (such as a genetic algorithm or sequential quadratic programming) is used to optimize the global objective function while satisfying physical constraints such as collision prevention for physical components. Offline joint optimization is performed to solve the problem. The final output is the globally optimal physical coordinate distribution. P and the optimal driving weight matrix W .

[0069] 4. Optimization Results and Technical Effects As shown in this embodiment Figure 3 As shown, after adopting this design method, although the physical light-emitting units are discretely distributed within the annular region, their positions... With weight Through optimization, the vectors of the same-color LEDs at the observation point were successfully synthesized into the expected equivalent vector. The three equivalent RGB vectors are nearly orthogonal in direction, have nearly equal magnitudes, and have a detection surface. The condition number distribution of the illumination matrix L on the surface approaches 1 ( Figure 4 With Mean=3.04, the illuminance distribution across all logic channels exhibits high uniformity (Max / Min approaches ideal values). This layout significantly improves the stability and accuracy of subsequent photometric stereo normal vector calculations.

[0070] Example 2: Joint optimization of light field for visual-tactile sensor based on confined rectangular ring.

[0071] The core principle of this embodiment is the same as that of embodiment 1. The main difference is that the geometric constraint of the sensor's mechanical structure is changed from a circle to a rectangular strip.

[0072] 1. Initial settings for physical space and light-emitting units: In this embodiment, K=3 logical lighting channels (R, G, B channels) are also constructed. The number of physical light-emitting units (RGB LEDs) in each channel is set to M1=6, M2=6, and M3=6, respectively.

[0073] Feasible layout area D feas The spatial coordinates P of the physical light-emitting unit are confined within a rectangular annulus. The inner rectangle has a side length of 20mm, and the outer rectangle has a side length of 30mm. Within this annulus, interference between the structure and the wiring is eliminated, allowing for a non-uniform distribution of the physical light source.

[0074] Region of Interest (ROI) The parameters of the detection surface are the same as those in Example 1 (a square with a height of 8mm and a side length of 10mm).

[0075] 2. Optimization of Solution and Result Evaluation: Reusing the aforementioned basic equivalent light vector synthesis formula and global objective function Nonlinear optimization is performed under the given rectangular boundary constraints. As shown in the attached figure of this embodiment... Figure 5 As shown, the algorithm automatically found the optimal position matrix that fits the rectangular ring. P and weight matrix W Under such stringent rectangular edge-emitting conditions, the system still successfully synthesized an orthogonal and uniform logic light field on the central detector surface through multi-source weighted joint driving: the average condition number of the illumination matrix was significantly reduced (see...). Figure 6 (Mean=2.98), the illuminance uniformity indices (Max / Min) for the R, G, and B channels reached 1.75, 1.54, and 2.00, respectively. This fully demonstrates the robustness and effectiveness of the light field synthesis method described in this invention in irregularly shaped and confined spaces.

[0076] This application embodiment also provides a visual-tactile sensor, including a physical light source, an optical element, a controller, and a sensing surface. Light emitted by the physical light source reaches the sensing surface through the optical element. A plurality of light-emitting units of the physical light source are installed according to the above-mentioned layout coordinates. The controller is used to control the light emission intensity of the plurality of light-emitting units of the physical light source according to the above-mentioned optimal driving weight. The optical element is determined according to the above-mentioned light transmission model.

[0077] In some embodiments, the modulation method for the optimal driving weight includes pulse width modulation or amplitude modulation.

[0078] The hardware foundation upon which the design method described in this invention relies mainly involves the illumination subsystem within a visual-tactile sensor. This subsystem, acting as the execution end for light field generation, is responsible for generating a specific illumination light vector field based on optimized parameters within a confined physical space.

[0079] Its core hardware mainly consists of the following three parts: (1) Distributed physical light source array Definition: Composed of several independent and controllable light-emitting units (such as LED particles or chips), it can be divided into K logic channels according to the nature of the light source, and the number of physical light-emitting units in each channel is indefinite.

[0080] Layout features: Spatial coordinates of the light-emitting unit within the device The arrangement is not a pre-fixed standard arrangement, but rather the optimized output of the design method described in this invention. The layout can be uniform or non-uniform, symmetrical or asymmetrical, depending on the structural constraints and optimization results.

[0081] Function: The intensity of each physical light-emitting unit is adjustable in response to driving commands.

[0082] (2) Optical shaping and light guiding structure Definition: An optical medium or structural component (such as a light guide, a light homogenizer, a reflective inner wall, or a freeform lens) located between a physical light source and a target sensing surface.

[0083] Function: This structure determines the physical light field from the light source. Transfer function to target point q .

[0084] Note: In the algorithm logic, the physical properties of the light guide structure (refractive index, geometry, etc.) are modeled as part of the light transmission model, directly affecting the equivalent light vector. The synthesis results.

[0085] Design considerations: This design method must fully consider the refraction, reflection and scattering of light by the light guide structure, and the optimization algorithm is performed under the optical boundary conditions defined by the structure.

[0086] (3) Multi-channel light vector synthesis controller Definition: A control module that includes embedded storage units and multiple independently controllable precision drive circuits.

[0087] Parameter carrier: The memory contains a static weight matrix calculated offline. W .

[0088] Intensity allocation execution: Controller based on W The system generates K×N parallel control signals. The driving circuit adjusts the luminous intensity of each physical light source according to these control signals.

[0089] Drive mode compatibility: This invention does not limit the specific dimming implementation method. The drive circuit can use pulse width modulation (PWM) to adjust the weight by adjusting the duty cycle. W Alternatively, DC / Analog Dimming can be used, where the weighting is achieved by adjusting the current amplitude. W .

[0090] Synchronous triggering: It has a synchronous triggering interface with an external imaging unit (Camera) to ensure that the physical light field is stably maintained in a specific equivalent light vector state during the image exposure window.

[0091] In one specific embodiment, see Figure 7 The diagram illustrates the principle of illumination vector synthesis for an illumination logic channel composed of multiple physical light-emitting units. 101, 102, and 103 represent illumination logic channel 1, and 201 and 202 represent illumination logic channel 2 (this diagram only illustrates the case where K=2 and does not represent the actual number of channels calculated by PS). 301 represents the optical shaping and light-guiding structure, and 401 represents the touch material layer.

[0092] See Figure 8 This application also provides a light source design device for a visual-tactile sensor. The visual-tactile sensor includes a physical light source, which includes several light-emitting units with several logical illumination channels. The device includes: The first module is used to acquire the sensor mechanical model and extract feasible layout regions and optical transmission models from the sensor mechanical model. The second module is used to determine the global objective function by combining the physical light source layout and driving weight parameters of the feasible layout area with the optical transmission model. The third module is used to solve the global objective function according to the preset evaluation index, and to determine the layout coordinates of the physical light source and the optimal driving weight; the evaluation index includes minimizing the condition number and / or maximizing uniformity.

[0093] It is understood that the content of the above method embodiments is applicable to the present device embodiments. The specific functions implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0094] This application also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the above-described method. This electronic device can be any smart terminal, including tablet computers, in-vehicle computers, etc.

[0095] It is understood that the content of the above method embodiments is applicable to this device embodiment. The specific functions implemented by this device embodiment are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0096] Please see Figure 9 , Figure 9 The hardware structure of an electronic device according to another embodiment is illustrated. The electronic device includes: The processor 901 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application. The memory 902 can be implemented as a read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM). The memory 902 can store the operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 902 and is called and executed by the processor 901 using the methods described in the embodiments of this application. The input / output interface 903 is used to implement information input and output; The communication interface 904 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.). Bus 905 transmits information between various components of the device (e.g., processor 901, memory 902, input / output interface 903, and communication interface 904); The processor 901, memory 902, input / output interface 903, and communication interface 904 are connected to each other within the device via bus 905.

[0097] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method.

[0098] It is understood that the content of the above method embodiments is applicable to this storage medium embodiment. The specific functions implemented in this storage medium embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0099] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0100] It is understood that the content of the above method embodiments is applicable to the embodiments of this program product. The specific functions implemented by the embodiments of this program product are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0101] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0102] The embodiments of this application include at least the following beneficial effects: This application provides a visual-tactile sensor and its light source design method and related equipment. This scheme extracts feasible layout regions and light transmission models through sensor mechanical models, determines a global objective function by combining the physical light source layout and driving weight parameters of the feasible layout regions with the light transmission model, solves the global objective function according to preset evaluation indicators, and determines the layout coordinates of the physical light source and the optimal driving weight. The evaluation indicators include minimizing the condition number and / or maximizing uniformity. By optimizing the spatial layout of the physical light source and the driving weight parameters in a coordinated manner, the consistency and signal-to-noise ratio of the illumination light field are improved, thereby optimizing the illumination light field and improving the reconstruction accuracy.

[0103] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0104] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0105] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0106] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0107] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0108] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0109] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0110] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0111] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0112] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0113] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A method for designing a light source for a visual-tactile sensor, characterized in that, The visual-tactile sensor includes a physical light source, which comprises several light-emitting units with several logical illumination channels. The method includes the following steps: Obtain the sensor mechanical model, and extract the feasible layout region and optical transmission model from the sensor mechanical model; The global objective function is determined by combining the physical light source layout and driving weight parameters of the feasible layout region with the optical transmission model. The global objective function is solved according to the preset evaluation index to determine the layout coordinates of the physical light source and the optimal driving weight; the evaluation index includes minimizing the condition number and / or maximizing uniformity.

2. The method according to claim 1, characterized in that, The determination of the global objective function by combining the physical light source layout and driving weight parameters of the feasible layout region with the optical transmission model includes: The transmission function is determined based on the light transmission model from the light-emitting unit at a specified position in the feasible layout area to the specified target point on the sensing surface of the visual-touch sensor in the preset logical lighting channel. The synthetic light vector of the specified target point is determined based on the transfer function and driving weight parameters; A global objective function is formed based on the synthesized light vectors of multiple specified target points on the sensing surface.

3. The method according to claim 1, characterized in that, The step of solving the global objective function according to the preset evaluation index to determine the layout coordinates of the physical light source and the optimal driving weight includes: A region of interest is determined on the sensing surface of the visual-touch sensor, and a global average conditional function for all target points within the region of interest is determined based on the global objective function. Minimize the global average condition function to determine the layout coordinates of the physical light source and the optimal driving weight.

4. The method according to claim 1, characterized in that, The step of solving the global objective function according to the preset evaluation index to determine the layout coordinates of the physical light source and the optimal driving weight includes: The ratio function between the maximum and minimum composite light vectors among all target points on the sensing surface is determined based on the global objective function. With the preset evaluation index being the maximization of uniformity, the layout coordinates of the physical light source and the optimal driving weight are determined by minimizing the proportional function.

5. The method according to claim 1, characterized in that, The step of solving the global objective function according to the preset evaluation index to determine the layout coordinates of the physical light source and the optimal driving weight includes: A region of interest is determined on the sensing surface of the visual-touch sensor, and a global average conditional function for all target points within the region of interest is determined based on the global objective function. The vector ratio function between the maximum and minimum composite light vectors among all target points on the sensing surface is determined based on the global objective function. The global cost function is determined based on the global average condition function, the vector scale function, and the preset weights. The layout coordinates of the physical light source and the optimal driving weights are determined by minimizing the global cost function.

6. A visual-tactile sensor, characterized in that, The device includes a physical light source, optical elements, a controller, and a sensing surface. Light emitted from the physical light source passes through the optical elements to reach the sensing surface. The plurality of light-emitting units of the physical light source are installed according to the layout coordinates of any one of claims 1-5. The controller is used to control the luminous intensity of the plurality of light-emitting units of the physical light source with the optimal driving weights according to any one of claims 1-5. The optical elements are determined by the light transmission model according to any one of claims 1-5.

7. The visual-tactile sensor according to claim 6, characterized in that, The modulation method for the optimal driving weight includes pulse width modulation or amplitude modulation.

8. A light source design device for a visual-tactile sensor, characterized in that, The visual-tactile sensor includes a physical light source, which comprises several light-emitting units with several logical illumination channels. The device includes: The first module is used to acquire a sensor mechanical model and extract a feasible layout region and an optical transmission model from the sensor mechanical model. The second module is used to determine the global objective function by combining the physical light source layout and driving weight parameters of the feasible layout area with the optical transmission model. The third module is used to solve the global objective function according to the preset evaluation index, and determine the layout coordinates of the physical light source and the optimal driving weight; the evaluation index includes minimizing the condition number and / or maximizing uniformity.

9. An electronic device, characterized in that, include: At least one processor; At least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the method as described in any one of claims 1-5.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 5.

Images