Imaging distance measurement method, system and device and composite sensor

Abstract

The invention discloses an imaging distance measurement method, system and device and a composite sensor. The imaging distance measurement method comprises the following steps: determining a working mode currently required by the composite sensor; the working mode comprises an imaging mode, a distance measurement mode or an imaging and distance measurement alternating mode; and controlling the composite sensor to be switched to a currently required working mode, and performing data acquisition on pixels corresponding to the working mode in the composite sensor.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to an imaging and ranging method, system, device and composite sensor. BACKGROUND

[0002] As a core technology for perceiving the environment and obtaining target information, image processing has deeply penetrated into many key fields such as intelligent driving and transportation, robots, security monitoring, industrial detection, virtual reality and augmented reality, and weapons and equipment, and has become an important support for promoting the intelligent upgrading of various industries.

[0003] However, the existing image processing still has significant limitations and cannot fully meet the demand for comprehensive performance in actual applications: traditional imaging devices only have a single image acquisition function and cannot simultaneously obtain distance information of a target object, resulting in insufficient spatial perception ability of the target.

[0004] In order to improve the spatial perception ability, the following schemes are mainly included: a binocular vision ranging scheme based on visible light can realize the combination of imaging and ranging, but the installation and calibration precision of the hardware device is extremely high, and the calculation complexity of image matching and distance solving is large, which directly causes the real-time performance of the ranging result output to be low, and it is difficult to adapt to scenes sensitive to response speed; a ranging scheme based on an additional laser ranging module has the advantages of high precision, long measurement distance and fast measurement speed, but it is difficult to be highly integrated with the imaging system, and it needs to rely on additional sensors, signal processing circuits and other components, which not only increases the device size and cost, but also reduces the use flexibility of the system.

[0005] Therefore, there is an urgent need for an imaging and ranging method that has high integration, low cost, high precision, high flexibility and high real-time performance. SUMMARY

[0006] To solve the existing technical problems, the present application provides an imaging and ranging method and device, a computer program product and a computer readable storage medium that have high integration, low cost, high precision, high flexibility and high real-time performance.

[0007] To achieve the above purpose, the technical scheme of the embodiments of the present application is as follows: In a first aspect, the embodiments of the present application provide a composite sensor, which includes a pixel array composed of a plurality of pixels, the pixel array includes imaging pixels and ranging pixels, and the structure of each pixel is the same and can be independently configured as an imaging function or a ranging function.

[0008] In a second aspect, the embodiments of the present application provide an imaging and ranging method for a composite sensor, which includes imaging pixels and ranging pixels, and the imaging and ranging method includes: determine a current required working mode of the composite sensor; the working mode comprises an imaging mode, a ranging mode, or an imaging and ranging alternation mode; control the composite sensor to switch to the current required working mode, so that pixels in the composite sensor corresponding to the working mode perform data acquisition.

[0009] In a third aspect, an embodiment of the present application provides an imaging and ranging system, which comprises a composite sensor and a master control system; the master control system is in communication connection with the composite sensor, and the master control system is configured to: determine a current required working mode of the composite sensor; the working mode comprises an imaging mode, a ranging mode, or an imaging and ranging alternation mode; control the composite sensor to switch to the current required working mode, so that pixels in the composite sensor corresponding to the working mode perform data acquisition.

[0010] In a fourth aspect, an embodiment of the present application provides an imaging and ranging device, which comprises a memory and a processor, and the memory stores a computer program; the computer program is executed by the processor to implement the imaging and ranging method in any of the embodiments of the present application.

[0011] The composite sensor provided in the above embodiments comprises a pixel array composed of a plurality of pixels, and the pixel array is specifically divided into two types of functional pixels, namely imaging pixels and ranging pixels; each pixel in the pixel array adopts a completely same hardware structure design, and each pixel has independent functional configuration capability, and can be flexibly configured to perform imaging function for imaging task, or perform ranging function for ranging task, or alternately perform imaging function for imaging task and ranging function for ranging task, to realize dual-function flexible adaptation on the basis of single pixel hardware.

[0012] The imaging and ranging method provided in the above embodiments is applied to the composite sensor, and in specific implementation, first, a target working mode currently required by the composite sensor is determined according to actual application scene requirements, such as ambient light conditions, measurement accuracy requirements, data acquisition efficiency demands, etc.; the target working mode specifically comprises an imaging mode for separately implementing imaging function, a ranging mode for separately implementing ranging function, and an imaging and ranging alternation mode for alternately executing imaging function and ranging function in cycles; then, the composite sensor is controlled to complete mode switching based on the target working mode, so that pixels in the composite sensor corresponding to the target working mode perform corresponding image data or ranging data acquisition. The image data and the ranging data are both acquired by the pixel array of the composite sensor, and have the same reference position, so that higher accuracy and higher real-time performance can be achieved; by switching the working mode of the composite sensor and processing data in a targeted manner, different use requirements can be adapted, and based on the integration of the composite sensor and the unified control logic, the hardware cost can be reduced, the operation is flexible, and the practicality is relatively strong.

[0013] In the above embodiments, the imaging ranging system and device and the corresponding imaging ranging method embodiments belong to the same concept, thus having the same technical effects as the human body detection and early warning method embodiments, which will not be described here. BRIEF DESCRIPTION OF DRAWINGS

[0014] Figure 1 Schematic diagram of an optional application scenario of the imaging ranging method; Figure 2 Schematic diagram of the structure of an imaging ranging device in an embodiment; Figure 3 Flowchart of the imaging ranging method in an optional specific example; Figure 4 Schematic diagram of a pixel array arrangement in a checkerboard interlaced arrangement; Figure 5 Schematic diagram of a pixel array arrangement in a block grouping arrangement; Figure 6 Schematic diagram of a pixel array arrangement in a concentric circular arrangement; Figure 7 Schematic diagram of a pixel array arrangement in a row-column separation arrangement; Figure 8 Schematic diagram of a pixel array arrangement in a random hash arrangement; Figure 9 Schematic diagram of a pixel array arrangement in a pixel combination arrangement. DETAILED DESCRIPTION

[0015] The technical solutions of the present application will be further described in detail below in combination with the accompanying drawings and specific embodiments.

[0016] In order to make the purposes, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in combination with the accompanying drawings. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by those of ordinary skill in the art without making creative efforts fall within the scope of protection of the present application.

[0017] In the following description, the expression “some embodiments” describes a subset of all possible embodiments, and it should be noted that “some embodiments” can be the same subset or different subsets, and can be combined with each other without conflict.

[0018] In the following description, the terms "first", "second", "third", etc. are merely used to distinguish similar objects, and do not represent a specific order or sequence of the objects. Understandably, the "first", "second", "third" can be interchanged in a specific order or sequence as allowed, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein.

[0019] The embodiments of the present application provide a composite sensor, comprising a programmable pixel array composed of a plurality of pixels, the pixel array comprising imaging pixels and ranging pixels, each pixel having the same structure and being independently configurable as an imaging function or a ranging function.

[0020] The composite sensor provided by the above embodiments comprises a pixel array composed of a plurality of pixels, and the pixel array is specifically divided into two types of functional pixels, namely imaging pixels and ranging pixels. Each pixel in the pixel array adopts a completely same hardware structure design, and each pixel has an independent function configuration capability. The imaging function for performing an imaging task or the ranging function for performing a ranging task can be flexibly configured according to the requirements of the actual application scene, or the imaging function and the ranging function are alternately executed. The dual-function flexible adaptation on the basis of single-pixel hardware is realized. Specifically, the configuration mode can be that a user changes a pixel configuration word in a control logic, a corresponding configuration signal is generated by the control logic, and the function configuration of each pixel is modified to switch the imaging and ranging functions.

[0021] Please refer to Figure 1An optional application scenario of the imaging and ranging method provided by the embodiment of the present application is shown in the figure, wherein the imaging and ranging system comprises a master control system 13, a composite sensor 11, a light source module 14 and a display output module 12 which are in communication connection with the master control system 13. The master control system 13 mainly comprises a memory and a processor. The imaging and ranging system is loaded with a computer program for executing the imaging and ranging method provided by the embodiment of the present application. The imaging and ranging system can be installed on a vehicle, an outdoor detection device or an industrial detection device carrier, etc. In the scenes of vehicle driving, outdoor personnel observing targets or industrial pipeline detecting parts, etc., the imaging and ranging functions can be correspondingly completed by executing the imaging and ranging method. In an optional example, the composite sensor 11 emits laser light according to the preset laser emission strategy under the control of the master control system 13 in cooperation with the light source module 14, and collects imaging light signals (for imaging) and laser reflection signals (for ranging) of the surrounding scene in real time, to form scene imaging data and original ranging data respectively, and synchronously sends the two types of data to the master control system 13. The processor of the master control system 13 cooperatively analyzes the scene imaging data and the original ranging data. On the one hand, the scene imaging data is processed for noise reduction and target recognition to determine the target objects in the surrounding, such as other vehicles, pedestrians, outdoor wild animals and industrial parts; on the other hand, the original ranging data is processed for time difference calculation and accuracy calibration to obtain the real-time distance between the target objects and the imaging and ranging system or the vehicle or the detection device. If it is determined by analysis that the distance between the target object and the imaging and ranging system is less than a safety threshold, such as less than 50 meters in the vehicle scene or less than 10 centimeters in the industrial scene, or there is a risk of rapid distance shortening, the master control system 13 sends a control instruction to the display output module 12 to control the display output module 12 to display the clear image and real-time distance of the target object, and synchronously sends a warning information, such as an audible and visual alarm or a screen pop-up window, to help the user, such as a driver, a detection personnel or an industrial operator, to know the target state and distance risk in advance and take measures such as avoiding, adjusting the observation angle or pausing the pipeline in time, to ensure the operation safety or detection accuracy.

[0022] Please refer to Figure 2 The imaging and ranging method provided by the embodiment of the present application comprises the following steps: S101, determining a current required working mode of the composite sensor; the working mode comprises an imaging mode, a ranging mode or an imaging and ranging alternating mode.

[0023] S103, controlling the composite sensor to switch to the current required working mode, so that the pixels in the composite sensor corresponding to the working mode perform data collection.

[0024] The imaging and ranging method provided by the above embodiment is applied to the composite sensor, and in specific implementation, first, a target working mode currently required by the composite sensor is determined according to requirements of an actual application scene, such as ambient light conditions, measurement accuracy requirements, data acquisition efficiency requirements, and the like, the target working mode specifically includes an imaging mode for separately implementing an imaging function, a ranging mode for separately implementing a ranging function, and an imaging and ranging alternation mode for alternately implementing the imaging function and the ranging function according to a period; then, the composite sensor is controlled to complete mode switching based on the target working mode, and then the pixels in the composite sensor that are adapted to the target working mode perform acquisition of corresponding image data or ranging data. The image data and the ranging data are both acquired directly by the pixel array of the composite sensor, and have the same reference position, so that higher accuracy and higher real-time performance can be achieved, different use requirements can be adapted by switching the working mode of the composite sensor and processing data in a targeted manner, and based on the integration of the composite sensor and the unified control logic, the hardware cost can be reduced, the operation is flexible, and the practicality is relatively strong.

[0025] The imaging and ranging method provided by the embodiment of the application is suitable for scenes that need to consider both imaging and ranging functions, such as security monitoring, outdoor detection, industrial detection, and the like. By controlling the composite sensor to switch to a target working mode and collecting data in the corresponding mode, the adaptability of the device to different application scenes and the accuracy and practicality of data output are effectively improved.

[0026] Optionally, after step S103, the imaging and ranging method further includes: processing the data collected by the composite sensor to form a data result required in the corresponding working mode.

[0027] The data collected by the composite sensor refers to original data collected by the composite sensor in a specified working mode, in the embodiment, the composite sensor mainly implements imaging and ranging functions, and the original data collected accordingly can be data used for subsequent implementation of the imaging and ranging functions, for example, can be an electrical signal converted from an external physical signal such as an optical signal, to provide a basis for subsequent data processing to implement the imaging and ranging functions. Processing the data can be operation, screening, integration, signal conversion, and the like on the data collected by the composite sensor, to achieve an output result meeting use requirements.

[0028] The composite sensor operates in three modes: imaging, ranging, and alternating imaging and ranging. In imaging mode, the data output includes image data, directly presenting the scene within the composite sensor's field of view, fulfilling visual observation needs. In ranging mode, the data output includes ranging data, typically presented numerically as the distance between the target and the composite sensor, meeting the requirements for precise measurement. In alternating imaging and ranging mode, the data output includes image data containing ranging information, presenting both the scene and overlaying distance markers (e.g., the distance to the target point), simultaneously satisfying both observation and measurement needs.

[0029] The system receives raw data collected by the composite sensor in its corresponding operating mode, such as imaging light signals in imaging mode, reflected laser signals in ranging mode, and dual raw signals in alternating imaging and ranging mode. It then performs targeted data processing based on the functional requirements of each operating mode. In imaging mode, the collected light signals are converted into electrical signals suitable for subsequent processing. After noise reduction, signal amplification, and format conversion, these electrical signals are processed to generate image data that clearly presents the target object and scene details—this is the data display result in imaging mode. In ranging mode, the laser signals carrying distance information are converted into electrical signals that can be recognized and processed by the electronic device. These electrical signals undergo noise reduction, intensity calibration, and data calculation to obtain numerical information reflecting the distance between the target object and the composite sensor—this is the distance data—the data display result in ranging mode. In alternating imaging and ranging mode, after the imaging light signals and laser signals are collected, they are converted into electrical signals that can be processed by the electronic device and processed differently. Finally, image data and distance data are output simultaneously, together constituting the data display result in alternating imaging and ranging mode.

[0030] Optionally, step S103 includes: When it is determined that the current required working mode of the composite sensor is imaging mode, an imaging mode signal is sent to the composite sensor to control the composite sensor to switch to imaging mode, so that the imaging pixels in the composite sensor are activated to collect the imaging light signal reflected by the target object. When it is determined that the current required working mode of the composite sensor is the ranging mode, a ranging mode signal is sent to the composite sensor to control the composite sensor to switch to the ranging mode, so that the ranging pixels in the composite sensor are activated to collect the laser signal reflected by the target object. When it is determined that the current required operating mode of the composite sensor is the alternating imaging and ranging mode, an alternating imaging and ranging mode signal is sent to the composite sensor to control the composite sensor to switch to the alternating imaging and ranging mode, so that the imaging pixels and ranging pixels in the composite sensor are alternately activated according to a set period to collect the imaging light signal and laser signal reflected by the target object, respectively.

[0031] The imaging mode signal, ranging mode signal, and alternating imaging-ranging mode signal are operating mode signals, primarily used to determine which operating mode the composite sensor should enter. These signals are used to adjust and control the composite sensor to acquire data in the expected operating mode, obtaining raw data in real time. Pre-defined calculations and processing are then performed to generate the corresponding data display results. Specifically, when the operating mode is set to imaging mode, the data display results include image data that meets visual observation requirements; when the operating mode is set to ranging mode, the data display results include ranging data that accurately determines the target distance; and when the operating mode is set to alternating imaging-ranging mode, the data display results include image data that incorporates ranging information, enabling simultaneous scene observation and distance measurement.

[0032] Based on the imaging requirements of the actual application scenario, when it is determined that the current working mode to be executed by the composite sensor is imaging mode, an imaging mode signal will be generated and sent to the composite sensor to switch the composite sensor to imaging mode. In imaging mode, the imaging pixels in the composite sensor acquire the imaging light signal reflected by the target object, and then convert the acquired light signal into an electrical signal that can be processed subsequently. After processing such as noise reduction, signal amplification, and format conversion, the final image data that clearly presents the target object and scene details is generated, which is the data display result corresponding to the imaging mode.

[0033] Based on the imaging requirements of the actual application scenario, when it is determined that the current operating mode of the composite sensor needs to be ranging mode, a ranging mode signal will be generated and sent to the composite sensor, causing the composite sensor to switch to ranging mode. In ranging mode, the ranging pixels of the composite sensor collect the laser signal reflected back from the target object, and then convert these laser signals carrying distance information into electrical signals that can be recognized and processed by electronic devices. Subsequently, through signal noise reduction, intensity calibration, data calculation, and other processing operations on the electrical signal, numerical information reflecting the distance between the target object and the composite sensor is obtained, that is, the distance data, which is the corresponding data display result in ranging mode.

[0034] Based on the requirement for synchronous acquisition of imaging and ranging data in real-world scenarios, when it is determined that the composite sensor needs to execute an alternating imaging and ranging mode, an alternating imaging and ranging mode signal will be generated and sent to the composite sensor, causing it to switch to this mode. The imaging and ranging pixels of the composite sensor alternately acquire data according to a set cycle. During the corresponding cycle, the imaging pixels acquire the imaging light signal reflected by the target object, including but not limited to visible light, infrared, and low-light signals. During the other cycle, the ranging pixels acquire the laser signal reflected by the target object. After the acquisition of both the imaging light signal and the laser signal, they are converted into electrical signals that can be processed by electronic devices. The system performs differentiated processing on these two electrical signals. For the electrical signal converted from the imaging light signal, noise reduction and image quality optimization are performed to generate image data that can be used to represent the target's shape. For the electrical signal converted from the laser signal, distance calculation and accuracy calibration are performed to generate distance data representing the distance between the target and the composite sensor. Finally, both image data and distance data are output simultaneously, together forming the data display result in the alternating imaging and ranging mode.

[0035] Optionally, step S101 includes: The initial operating mode of the composite sensor is determined based on the initialization command, and the initial operating mode is used as the currently required operating mode; or, the currently required operating mode of the composite sensor is determined based on the received operating mode configuration command.

[0036] In the above embodiments, determining the current required operating mode of the composite sensor includes, but is not limited to, the following two implementation methods: The first method involves obtaining an initialization command from the system or device equipped with the composite sensor. Based on this command, the initial operating mode required for the composite sensor to start is determined. A corresponding operating mode signal is then generated and sent to the composite sensor, completing the initial configuration of its operating parameters. For example, when a home security camera equipped with a composite sensor is powered on, the main control chip inside the camera performs a self-test. After confirming that the device is functioning correctly, it immediately sends an initialization command to the composite sensor, instructing it to operate in a preset mode, such as imaging mode. This allows the camera to display an image immediately upon power-on.

[0037] The second method involves obtaining corresponding operating mode configuration commands through user operations via buttons, configuration interfaces, or other means. Based on the operating mode configuration, the user-selected target operating mode is determined, and a corresponding operating mode signal is generated and sent to the composite sensor. This enables manual switching of the composite sensor's operating mode, matching the user's real-time needs. Specifically, operation via the configuration interface can involve selecting the operating mode through touchscreen operation on the display screen. Other methods include, but are not limited to, existing or future feasible operation forms such as voice control and wireless control.

[0038] The two methods mentioned above can be used individually or in combination as needed, and this application does not impose any restrictions on this.

[0039] Optionally, when the current required operating mode of the composite sensor is ranging mode or alternating imaging ranging mode, the imaging ranging method further includes: Determine the required laser emission strategy and control the light source module to emit laser light toward the target object according to the laser emission strategy.

[0040] Based on the requirements of the actual application scenario, the current working mode of the composite sensor is determined to be either ranging mode or an alternating imaging and ranging mode. The imaging ranging method also includes laser emission. First, considering the core requirements of the current working mode, such as measurement accuracy, ambient light conditions, and target object distance range in ranging mode, a suitable laser emission strategy is comprehensively analyzed and determined. This laser emission strategy includes, but is not limited to, laser emission power, emission angle, and emission field of view. Based on the laser emission strategy, corresponding control commands are generated to control the light source module to emit laser light towards the target object according to the laser emission strategy, providing basic light source support for the ranging pixels in the composite sensor to subsequently receive reflected laser signals and calculate the target object distance.

[0041] Optionally, determining the currently required laser emission strategy includes: Read the ranging pixel configuration information in the composite sensor and match the corresponding laser emission strategy according to the ranging pixel configuration information; or, receive the laser parameters set by the user and determine the corresponding laser emission strategy according to the laser parameters.

[0042] The specific implementation methods for determining the required laser emission strategy include at least the following two: First, the memory pre-stores a preset mapping relationship between ranging pixel configuration information and laser emission strategies. This preset mapping relationship records the configuration parameters such as the number and arrangement of different ranging pixels and the matching laser emission strategies, including but not limited to the correspondence between laser emission power, emission angle, and emission field of view. During operation, the preset ranging pixel configuration information in the composite sensor, such as the number and arrangement of ranging pixels, is read. Based on the preset mapping relationship between the ranging pixel configuration information and the laser emission strategy, a laser emission strategy matching the ranging pixel configuration is automatically matched. Second, the user manually sets laser parameters through a configuration interface, buttons, etc., including but not limited to laser emission power, emission angle, and emission field of view. Based on these user-defined laser parameters, the laser emission strategy is parsed and generated, and finally, a laser emission strategy that meets the user's needs is determined. The two methods can be flexibly selected according to the actual application scenario, and can be used alone or in combination. Specifically, setting laser parameters through the configuration interface can be done manually via touchscreen operation on the display interface. Other methods include, but are not limited to, existing or future feasible operation forms such as voice control and wireless control.

[0043] Optionally, the imaging ranging method further includes: The required pixel array mode for the composite sensor is determined, and the functions of each pixel in the composite sensor are configured according to the pixel array mode so that the imaging pixels and ranging pixels in the composite sensor are arranged according to the pixel array mode.

[0044] Pixel array mode refers to the pixel array form formed by the number and position of imaging pixels and ranging pixels in a composite sensor, which consists of multiple pixels. Composite sensors include m... Taking an n-pixel array as an example, different pixel array patterns can correspond to m In an n-pixel array, M pixels are imaging pixels and N pixels are ranging pixels, and the M imaging pixels are in the range of m. The position of the n-pixel array and the N ranging pixels in m Position in an n-pixel array.

[0045] Composite sensors integrate imaging and ranging functions. First, based on the current usage scenario, such as high-precision outdoor ranging or imaging in low-light environments, a suitable pixel array mode must be determined. Then, based on this mode, specific functions are assigned to each pixel in the sensor. Ultimately, the imaging and ranging pixels within the sensor are arranged in an orderly manner according to the preset pixel array mode, achieving coordinated adaptation of imaging and ranging functions. Different scenarios have different requirements for imaging quality and ranging accuracy, and the corresponding pixel array modes also differ. For example, high-precision ranging scenarios may allocate more pixels as ranging pixels, while high-resolution imaging scenarios allocate more imaging pixels. By first determining the pixel array mode and then configuring the pixel functions, composite sensors can flexibly adapt to different usage needs while maintaining the core performance of both imaging and ranging.

[0046] Optionally, determining the pixel array pattern currently required by the composite sensor includes: The initial pixel array mode of the composite sensor is determined based on the initialization command, and the initial pixel array mode is used as the currently required pixel array mode; or, the currently required pixel array mode of the composite sensor is determined based on the received pixel array configuration command.

[0047] Pixel array mode configuration refers to the parameter settings used to determine the pixel array mode of the composite sensor. Specifically, the pixel array mode can include the number, proportion, and arrangement of different types of pixels in the pixel array. In a specific example, the different types of pixels mainly include two categories: imaging pixels used for imaging and ranging pixels used for ranging. The selectable pixel array modes include, but are not limited to, the following: Figure 4 The chessboard grid is arranged in an alternating pattern. Figure 5 Block-based grouping arrangement Figure 6 concentric ring arrangement Figure 7 Row and column separation arrangement Figure 8 Random hash permutation, Figure 9 Macro-pixel combinations, multi-layer stacked arrangements, and programmable pixel arrangements are all possible. It should be noted that selecting different pixel array modes can change the number and position of the imaging and ranging pixels that perform imaging and ranging functions respectively in the corresponding operating mode of the composite sensor. This can correspondingly adjust the resolution of the image data and the accuracy of the ranging data obtained by the composite sensor in the corresponding operating mode.

[0048] Determining the appropriate pixel array mode for a composite sensor can be achieved through at least the following two options: The first method involves obtaining an initialization command from the system or device equipped with the composite sensor. Based on this command, the initial pixel array mode (i.e., the preset arrangement rule of imaging pixels and ranging pixels) to be used when the composite sensor starts up is determined. The corresponding pixel array mode configuration signal is then generated and sent to the composite sensor according to the determined initial pixel array mode, completing the configuration of the initial pixel array function allocation for the composite sensor. For example, when an outdoor industrial inspection device equipped with a composite sensor is powered on, the main control module inside the device first completes self-test and system initialization. After confirming that there are no abnormalities, it sends an initialization command to the composite sensor, without changing the pixel array mode by default. This allows the device to quickly respond to the high-precision ranging requirements in industrial scenarios after startup, while also taking into account basic imaging and positioning functions.

[0049] The second method involves obtaining corresponding pixel array configuration instructions through user operations such as device button presses, the accompanying APP configuration interface, and remote control commands. Based on these instructions, the user-selected target pixel array mode is determined, such as a checkerboard pattern. A corresponding pixel array mode switching signal is generated and sent to the composite sensor. Alternatively, each pixel unit in the pixel array matrix can be configured, explicitly designating some pixels for imaging and others for ranging. This allows for manual adjustment of the composite sensor's pixel arrangement to precisely match the user's real-time needs. For example, in the local configuration interface of an outdoor industrial inspection device, for a short-range high-definition imaging + low-precision ranging scenario, the user manually selects a checkerboard pattern. After obtaining this pixel array configuration instruction, the system immediately sends a pixel array mode switching signal to the composite sensor, adjusting the pixel array arrangement to output a clearer image of the target detail while meeting basic ranging requirements. Both methods can be flexibly selected based on the actual application scenario; they can be used individually or in combination.

[0050] Please see Figure 3 To provide a more comprehensive understanding of the imaging ranging method provided in the embodiments of this application, an optional specific example is used to illustrate the imaging ranging method. The imaging ranging system includes a composite sensor, a light source module, a display output module, and a main control system, wherein the main control system includes a memory and a processor.

[0051] The following embodiments use a gating signal triggering mechanism as an example to illustrate the imaging ranging method. The gating signal is used in this embodiment to trigger the switching of the composite sensor's working mode and the start / stop of the light source module. The high and low level states and duration ratio of the gating signal determine the working mode switching logic of the composite sensor and the coordination mechanism of each module. Subsequently, the dynamic adaptation of imaging and ranging functions will be realized based on the timing rules of this gating signal.

[0052] S11. Configure the processor's triggering rules and linkage mechanisms based on user needs and usage scenarios. The processor is the central hub for system operation rules, requiring specific configuration of the timing triggering rules for the gating signal and the linkage parameters between the processor and the composite sensor and light source module. For example, a high level gating signal might trigger the composite sensor to switch to ranging mode and the light source module to emit laser light, while a low level gating signal might trigger the composite sensor to switch to imaging mode and the light source module to stop working. Furthermore, different level percentage rules are set based on scenario characteristics to match the operating mode with scenario requirements. For instance, if the user requirement is to prioritize ranging during high-speed driving and focus on imaging at low speeds in a vehicle-mounted scenario, the processor can be configured with the following timing rules for the gating signal: when the vehicle speed is >60km / h, the high level duration of the gating signal accounts for 70% to match the ranging mode; when the vehicle speed is <30km / h, the low level duration of the gating signal accounts for 80% to adapt to the imaging mode. These configuration parameters and rules for the processor are stored in the main control system's memory as a benchmark for subsequent collaborative work among various modules.

[0053] S13, the composite sensor switches its operating mode and outputs image information and / or distance information under the gating signal output by the processor. The gating signal is a timing instruction that controls the mode switching of the composite sensor, typically alternating between high and low levels; for example, a low level corresponds to imaging mode, and a high level corresponds to ranging mode. When the gating signal is low, the composite sensor, according to the processor's preset rules, divides 70%-90% of the pixels into imaging pixels, arranging them in a checkerboard or row-column pattern to improve image detail capture, retaining only a small number of pixels for auxiliary positioning. At this time, the imaging pixels collect the imaging light signal reflected from the target, and after photoelectric conversion, generate an image electrical signal containing the target's contour, color, or grayscale information, i.e., image information. When the gating signal switches to high, the composite sensor automatically adjusts the pixel configuration, switching 50%-70% of the pixels to ranging pixels, arranging them in concentric circles or blocks, focusing the laser reflection signal. The ranging pixels receive the laser reflection signal emitted by the light source module and convert it into an electrical signal carrying time difference information, i.e., distance information.

[0054] The light source module adjusts its emission power, emission angle, and emission field of view under the control signals output by the processor to match the working mode of the composite sensor. The control signals are instructions sent by the processor to the light source module, containing specific parameter thresholds for laser emission. For example, when the composite sensor is in imaging mode, the control signal instructs the light source module to stop working, preventing laser interference with the imaging pixels' acquisition of natural light. When the composite sensor switches to ranging mode, the control signal adjusts the parameters. For instance, in a vehicle scenario where the target is a car 100 meters away, the emission power is increased to 30mW, the emission angle is narrowed to 5°, focusing on the target area, and the field of view matches the field of view of the composite sensor's ranging pixel array, ensuring sufficient laser reflection signal strength. In an industrial scenario where the target is an industrial component within 5 meters, the power is reduced to 10mW, the angle is widened to 15°, and the field of view covers the entire component, preventing signal saturation due to excessive power. Through dynamic parameter adjustment, the light source module always precisely matches the working mode of the composite sensor, ensuring ranging accuracy while avoiding energy waste or signal interference.

[0055] S15, the processor reads and processes the image information and / or distance information output by the composite sensor based on the gating signal and related control signals. First, the processor confirms the current operating mode through the gating signal, then, combined with the timing parameters in the control signal, accurately extracts the original information corresponding to the mode from the data stream output by the composite sensor. For image information, the processor performs noise reduction, contrast enhancement, and target contour extraction, converting the original electrical signal into recognizable image data, such as RGB format or grayscale. For distance information, the processor calculates the actual target distance based on the laser parameters in the control signal and the time difference of the laser recorded by the gating signal, using the formula "distance = speed of light × time difference ÷ 2," and calibrates the distance deviation based on the spatial distribution of the pixel array. After processing, the image data and distance data are correlated and integrated, such as marking the target position in the image and overlaying the distance value, forming structured and effective information.

[0056] S17, Image and / or distance information output; the integrated information processed by the processor is presented to the user through the display output module, with the output format adapted to the scenario requirements. For example, in a vehicle scenario, the information is displayed in real time on the dashboard or central control screen; in an industrial scenario, the information is synchronized to the production line monitoring terminal; in an outdoor detection scenario, the information is displayed on the device's built-in touch screen. In addition, some scenarios also support data interface output, such as transmission to terminal devices via USB or wireless for subsequent analysis or secondary processing. Through the display output module, users can intuitively obtain the visual characteristics and spatial distance of the target, providing direct basis for decision-making.

[0057] Please see Figure 1Another embodiment of this application provides an imaging ranging system, including: a main control system 210, a light source module 211, and a composite sensor 213. The main control system 210 includes a processor 214 and a memory 212. The memory 212 stores computer programs. The processor 214 sends mode signals to the composite sensor 213. The main control system 210 is communicatively connected to the composite sensor 213 and determines the current operating mode required by the composite sensor 213, sending a corresponding operating mode signal and a pixel array mode signal to the composite sensor 213. The composite sensor 213 receives the operating mode signal, configures a pixel array corresponding to the operating mode according to the operating mode signal, enters the corresponding operating mode, and collects data in the corresponding operating mode. The processor 214 also processes the collected data to obtain data display results in the corresponding operating mode. The light source module 211 emits laser light according to a corresponding laser emission strategy under the control of the processor 214.

[0058] It should be noted that the triggering mechanism for switching operating modes of the composite sensor 213 can be flexibly designed. One specific method is to achieve precise control through the level state of a gating signal. The gating signal, as an electrical signal used for timing or mode switching, has two basic states: low and high. These two states correspond to different operating modes of the system. In a specific example, when the gating signal is in a low-level state, the composite sensor 213 enters imaging mode; and when the gating signal switches to a high-level state, the composite sensor 213 enters ranging mode. This mechanism, which triggers mode switching through the level state of the gating signal, features rapid response and clear logic, allowing the composite sensor 213 to quickly switch between imaging and ranging functions, meeting the needs of scenarios with high real-time requirements.

[0059] Optionally, the processor 214 is the command center of the imaging ranging system, responsible for generating and sending working mode signals and pixel array mode signals according to user needs or preset rules, so that the composite sensor 213 enters the corresponding working state. For example, when a user wants to take a picture of a distant mountain peak outdoors and presses the imaging mode button on the camera, the processor 214 recognizes the operation and generates a mode signal containing the imaging mode and a checkerboard pixel arrangement; if the user later wants to measure the distance from the mountain peak to himself and switches to the ranging mode, the processor 214 will regenerate the ranging mode and the concentric circle pixel arrangement mode signal, and accurately send these signals to the composite sensor 213.

[0060] Optionally, the composite sensor 213 serves as the data acquisition center of the imaging ranging system. It receives the operating mode signal and pixel array mode signal from the processor 214 and completes two key configurations according to instructions: configuring the pixel array and switching to the corresponding operating mode. After configuration, the composite sensor 213 acquires data, capturing the imaging light signal or reflected laser signal of the target object. Furthermore, the composite sensor 213 has the function of flexibly configuring pixels. The pixel array can be configured with several preset pixel array arrangements before the product leaves the factory, or the user can freely set each pixel unit in the pixel array matrix as an imaging pixel or a ranging pixel. For example, when receiving an imaging mode signal, the composite sensor 213 automatically switches to a preset pixel array arrangement, setting 80% of the pixels as imaging pixels and 20% of the pixels for positioning assistance. At this time, the imaging pixels capture the natural light signal reflected from the mountain peak and convert it into an electrical signal. If a ranging mode signal is received, the composite sensor 213 will automatically switch to a preset pixel array arrangement, setting 60% of the pixels as ranging pixels and 40% of the pixels as auxiliary calibration pixels. At this time, the ranging pixels will capture the laser signal reflected by the mountain peak and convert it into an electrical signal.

[0061] Optionally, the number of processors 214 may include one or more. In this embodiment, there is only one processor 214, which may be an existing image processor in the infrared imaging device or another processor added to execute the computer program of the imaging ranging method described in this application embodiment. The processor 214 can serve as the data processing center of the imaging and ranging system, responsible for receiving the raw electrical signals collected by the composite sensor 213, and performing calculations, analysis, and conversions using preset algorithms. For example, it converts the electrical signals into image images or distance information, generating data display results for the corresponding working mode, allowing users to directly obtain effective information. For instance, in imaging mode, the processor 214 receives electrical signals from the imaging pixels, performs noise reduction, color calibration, and sharpness optimization, generating a clear photo of the mountain peak, which is displayed on the camera screen. In ranging mode, the processor 214 receives electrical signals from the ranging pixels, combines the difference between the laser emission time and the reflection signal reception time, and calculates the distance using the formula "distance = speed of light × time difference ÷ 2" to obtain the value "the mountain peak is 1200 meters from the current location," which is also displayed on the screen.

[0062] It should be noted that processor 214 should be interpreted broadly, referring to components in electronic devices responsible for data processing and logical operations, such as image processors (GPUs), central processing units (CPUs), and microcontrollers (MCUs). In another optional example, there may be multiple processors 214, including the original image processor in the infrared imaging device and an added controller. The image processor receives electrical signals from the imaging pixels of the composite sensor 213 and forms an image signal. The controller can load richer image processing algorithms to enhance the image obtained by the image processor or perform target recognition through image recognition algorithms. The controller and the image processor work together to implement the imaging ranging method described in any embodiment of this application.

[0063] Optionally, the light source module 211 is a laser emitter with a ranging function, responsible for emitting laser light towards the target object, including but not limited to laser sources such as 850nm, 905nm, and 940nm. After the laser light is reflected by the target, it is captured by the ranging pixels in the composite sensor 213. For example, when the camera enters ranging mode, the light source module 211, in conjunction with the composite sensor 213, emits a laser beam towards the mountain peak. After the laser light reaches the surface of the mountain peak, it is reflected back and captured by the ranging pixels of the composite sensor 213.

[0064] Another embodiment of this application provides an imaging ranging device, including: a memory and a processor, wherein the memory stores a computer program; when the computer program is executed by the processor, it implements the imaging ranging method described in any embodiment of this application.

[0065] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0066] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0067] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An imaging range finding method characterized by, An imaging and ranging method for a composite sensor comprising imaging pixels and ranging pixels, the method comprising: determining a current required working mode of the composite sensor; the working mode comprising an imaging mode, a ranging mode, or an imaging and ranging alternating mode; controlling the composite sensor to switch to the current required working mode, and enabling pixels corresponding to the working mode in the composite sensor to collect data.

2. The imaging rangefinding method of claim 1, wherein, Further comprising: processing the data collected by the composite sensor to form a data result required in the corresponding working mode.

3. The imaging ranging method of claim 1, wherein, The controlling the composite sensor to switch to the current required working mode, and enabling pixels corresponding to the working mode in the composite sensor to collect data, comprises: when it is determined that the current required working mode of the composite sensor is the imaging mode, sending an imaging mode signal to the composite sensor to control the composite sensor to switch to the imaging mode, and enabling imaging pixels in the composite sensor to collect imaging light signals reflected by a target object; when it is determined that the current required working mode of the composite sensor is the ranging mode, sending a ranging mode signal to the composite sensor to control the composite sensor to switch to the ranging mode, and enabling ranging pixels in the composite sensor to collect laser signals reflected by the target object; when it is determined that the current required working mode of the composite sensor is the imaging and ranging alternating mode, sending an imaging and ranging alternating mode signal to the composite sensor to control the composite sensor to switch to the imaging and ranging alternating mode, and enabling imaging pixels and ranging pixels in the composite sensor to be alternately activated according to a set period to respectively collect imaging light signals and laser signals reflected by the target object.

4. The imaging ranging method of claim 1, wherein, The determining the current required working mode of the composite sensor comprises: determining an initial working mode of the composite sensor based on an initialization instruction, and taking the initial working mode as the current required working mode; or determining the current required working mode of the composite sensor based on a received working mode configuration instruction.

5. The imaging range finding method of claim 1, wherein, when the current required working mode of the composite sensor is the ranging mode or the imaging and ranging alternating mode, the imaging and ranging method further comprises: determining a current required laser emission strategy, and controlling a light source module to emit laser to the target object according to the laser emission strategy.

6. The imaging ranging method of claim 5, wherein, The determining the current required laser emission strategy comprises: reading ranging pixel configuration information in the composite sensor, and matching a corresponding laser emission strategy according to the ranging pixel configuration information; or receiving laser parameters set by a user, and determining a corresponding laser emission strategy according to the laser parameters.

7. The imaging rangefinding method of claim 1, wherein, The imaging and ranging method further comprises: determining a current required pixel array mode of the composite sensor, and configuring functions of each pixel in the composite sensor according to the pixel array mode, so that imaging pixels and ranging pixels in the composite sensor are arranged according to the pixel array mode.

8. The imaging ranging method of claim 7, wherein, The determining the current required pixel array mode of the composite sensor comprises: determine an initial pixel array mode of the composite sensor based on the initialization instruction, and take the initial pixel array mode as a current required pixel array mode; or determine a current required pixel array mode of the composite sensor based on a received pixel array configuration instruction.

9. A composite sensor, characterized by The composite sensor comprises a pixel array composed of a plurality of pixels, and the pixel array comprises imaging pixels and ranging pixels, and each pixel has the same structure and can be independently configured as an imaging function or a ranging function.

10. The composite sensor of claim 9, wherein, The imaging pixels and the ranging pixels are arranged in a pixel array mode, and the pixel array mode comprises at least one of a chessboard interlaced arrangement, a block grouping arrangement, a concentric ring arrangement, a row-column separation arrangement, a random hash arrangement, a multi-layer stacking arrangement, a macro-pixel combination arrangement, and a programmable pixel arrangement.

11. An imaging range-finding system characterized by, The composite sensor comprises the composite sensor and the host system. The host system is in communication connection with the composite sensor, and the host system is used for: determining a current required working mode of the composite sensor; the working mode comprises an imaging mode, a ranging mode, or an imaging and ranging alternation mode; controlling the composite sensor to switch to the current required working mode, so that the pixels corresponding to the working mode in the composite sensor perform data acquisition.

12. The imaging range-finding system of claim 11, wherein, The composite sensor further comprises a light source module, and the light source module is used for emitting laser light according to a corresponding laser emission strategy under the control of the host system.

13. An imaging range-finding device characterized by comprising: The composite sensor comprises a memory and a processor, and the memory stores a computer program. The computer program is executed by the processor to implement the imaging and ranging method according to any one of claims 1 to 8.

Images