Method and apparatus for detecting deployment position of sensor, and vehicle

Abstract

Embodiments of the present application provide a method and apparatus for detecting the deployment position of a sensor, and a vehicle. The method comprises: acquiring a first image frame sequence, determining optical flow information of the first image frame sequence, then on the basis of the optical flow information of the first image frame sequence collected by an image sensor, detecting whether the deployment position of the image sensor matches the image sensor. Anti-reverse detection of an image sensor in a vehicle is achieved by means of software detection, and the deployment position of the image sensor collecting the first image frame sequence is detected on the basis of the optical flow information of the first image frame sequence, thereby improving the accuracy of anti-reverse detection.

Description

Sensor deployment location detection method, device, and vehicle

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on March 22, 2024, with application number 202410342469.7 and application name “Method, device and vehicle for detecting sensor deployment position”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of vehicle technology, and in particular to a method and device for detecting a sensor deployment position, and a vehicle. Background Art

[0003] In the field of intelligent driving, multiple image sensors deployed in a vehicle can capture images of the surrounding environment. These images can then be used for sensor calibration, visual perception, and other processing to achieve machine vision. Multiple image sensors can be used to capture images from different "perspectives" around the vehicle, such as forward-facing, rear-facing, and side-facing sensors. Sensors with different "perspectives" are deployed in different locations within the vehicle. Deploying an image sensor in the wrong location can lead to inaccurate sensor calibration and abnormal visual perception results.

[0004] Currently, hardware anti-reverse detection is commonly used to prevent sensors from being deployed in the wrong location. For example, image sensors deployed in different locations have different slots, preventing them from being inserted into the wrong slot. However, this solution requires different slots for different image sensors, increasing vehicle costs. Other hardware anti-reverse detection solutions also increase vehicle costs. Therefore, a solution for implementing anti-reverse detection for image sensor deployment is urgently needed. Summary of the Invention

[0005] The present application provides a sensor deployment position detection method, device, and vehicle, which implement anti-reflection of image sensors by detecting the deployment position of image sensors in the vehicle.

[0006] In a first aspect, the present application provides a method for detecting the deployment location of a sensor. The executing entity of the method may be a control device, which may be implemented as an electronic device, chip, chip system or other functional module in a vehicle that can call and execute a program.

[0007] The method includes: a control device acquires a first image frame sequence, determines optical flow information of the first image frame sequence, and then, based on the optical flow information of the first image frame sequence acquired by the image sensor, detects whether the deployment position of the image sensor matches the image sensor. Anti-reflection of the image sensor in the vehicle is achieved through software detection, and the deployment position of the image sensor that acquires the first image frame sequence is detected based on the optical flow information of the first image frame sequence, thereby improving the accuracy of anti-reflection detection.

[0008] It should be understood that when the image sensor is deployed at different positions in the vehicle, due to differences in the viewing angles of image acquisition, the optical flow information of the first image frame sequence acquired by the image sensor presents different characteristics.

[0009] In one possible implementation, the optical flow information includes the position of the optical flow vanishing point in the imaging plane of the image sensor. When the optical flow vanishing point is at different positions in the imaging plane of the image sensor, the deployment position of the image sensor can be distinguished. For example, if the optical flow vanishing point is in the middle area of ​​the imaging plane of the image sensor, the image sensor is deployed at the position of the front view sensor or the rear view sensor; or, if the optical flow vanishing point is in the left area of ​​the imaging plane of the image sensor, the image sensor is deployed at the position of the left rear view sensor or the right front view sensor; or, if the optical flow vanishing point is in the right area of ​​the imaging plane of the image sensor, the image sensor is deployed at the position of the left front view sensor or the right rear view sensor.

[0010] In one possible implementation, the optical flow information includes an optical flow direction. Different optical flow directions can distinguish different deployment locations of the image sensor. For example, if the optical flow direction diverges outward, the image sensor is deployed at the location of the front-view sensor, the left front-view sensor, or the right front-view sensor; or if the optical flow direction converges inward, the image sensor is deployed at the location of the rear-view sensor, the left rear-view sensor, or the right rear-view sensor.

[0011] In one possible implementation, the optical flow information includes optical flow velocity, and different optical flow velocities can be used to distinguish different deployment positions of the image sensor. For example, if the optical flow velocity is greater than or equal to a threshold, the image sensor is deployed at the location of the forward-looking telephoto sensor; or, if the optical flow velocity is less than the threshold, the image sensor is deployed at the location of the forward-looking wide-angle sensor.

[0012] In one possible embodiment, in order to improve the accuracy of the detection results, the control device can obtain the driving information of the vehicle and detect the deployment position of the image sensor in combination with the driving information of the vehicle. The driving information may include but is not limited to the driving speed and / or driving state of the vehicle, and the driving state of the vehicle includes but is not limited to at least one of execution, turning, or driving on a slope. Exemplarily, the control device can obtain the driving information of the vehicle, and when the driving information of the vehicle meets the preset conditions, the control device detects whether the deployment position of the image sensor matches the image sensor. For example, when the driving speed of the vehicle meets the speed threshold and the vehicle is traveling straight, the control device can detect whether the deployment position of the image sensor matches the image sensor based on the optical flow information.

[0013] In one possible embodiment, the control device can detect the deployment position of the image sensor based on optical flow information and determine whether the deployment position of the image sensor is the same as the target deployment position, where the target deployment position is the deployment position corresponding to the image sensor in the corresponding relationship. It should be understood that the corresponding relationship may include a correspondence between at least one image sensor and at least one deployment position, and this correspondence is used to indicate the target deployment position corresponding to each image sensor in the vehicle. Only when each image sensor is deployed at its corresponding target deployment position for image acquisition can the accuracy of sensor calibration and perception processing of the vehicle's machine vision system be ensured. Exemplarily, when the detected deployment position of the image sensor is the same as the target deployment position, the control device determines that the deployment position of the image sensor matches the image sensor, or that the deployment position of the image sensor is correct. When the detected deployment position of the image sensor is different from the target deployment position, the control device determines that the deployment position of the image sensor does not match the image sensor, or that the deployment position of the image sensor is incorrect.

[0014] In a possible implementation, in order to improve processing efficiency and reduce time and labor costs, the control device may update the above correspondence when detecting that the deployment position of the image sensor is incorrect.

[0015] Exemplarily, the control device may send an indication message to the deserializer, the indication message carrying the updated correspondence, the indication message being used to indicate that the information of the second image frame sequence deserialized by the deserializer corresponds to the deployment position of the image sensor, and the second image frame sequence is captured by the image sensor.

[0016] In one possible embodiment, in order to ensure the accuracy of the updated correspondence, the control device can obtain a second image frame sequence and determine the optical flow information of the second image frame sequence, and then detect whether the deployment position of the image sensor matches the image sensor based on the optical flow information of the second image frame sequence.

[0017] In one possible implementation, the control device may determine an update result based on a detection result of whether the deployment position of the image sensor matches the image sensor. If it is determined that the deployment position of the image sensor matches the image sensor, the update result indicates that the update is successful. For example, if the deployment position of the image sensor does not match the image sensor, the update result indicates that the update failed.

[0018] In a possible implementation, the control device may push the update result, for example, present the update result through a display screen, or send the update result to other devices.

[0019] In a second aspect, an embodiment of the present application provides a control device, comprising: an acquisition module for acquiring a first image frame sequence, wherein the first image frame sequence is acquired by an image sensor deployed in a vehicle; a processing module for determining optical flow information of the first image frame sequence; and a detection module for detecting, based on the optical flow information, whether the deployment position of the image sensor matches the image sensor.

[0020] In one possible embodiment, the optical flow information includes the position of the optical flow vanishing point in the imaging plane of the image sensor; the optical flow vanishing point is in the middle area of ​​the imaging plane of the image sensor, and the image sensor is deployed at the position of the front view sensor or the rear view sensor; or, the optical flow vanishing point is in the left area of ​​the imaging plane of the image sensor, and the image sensor is deployed at the position of the left rear side view sensor or the right front side view sensor; or, the optical flow vanishing point is in the right area of ​​the imaging plane of the image sensor, and the image sensor is deployed at the position of the left front side view sensor or the right rear side view sensor.

[0021] In one possible embodiment, the optical flow information includes an optical flow direction; the optical flow direction diverges outward, and the image sensor is deployed at the position of the front-view sensor, the left front-view sensor, or the right front-view sensor; or, the optical flow direction converges inward, and the image sensor is deployed at the position of the rear-view sensor, the left rear-view sensor, or the right rear-view sensor.

[0022] In one possible implementation, the optical flow information includes an optical flow velocity; if the optical flow velocity is greater than or equal to a threshold, the image sensor is deployed at the position of a forward-looking telephoto sensor; or, if the optical flow velocity is less than a threshold, the image sensor is deployed at the position of a forward-looking wide-angle sensor.

[0023] In a possible embodiment, the acquisition module is also used to obtain driving information of the vehicle; the processing module is also used to determine the optical flow information of the first image frame sequence based on the first image frame sequence when the driving information of the vehicle meets a preset condition; wherein the driving information includes the driving speed and / or driving status of the vehicle, and the driving status includes at least one of straight driving, turning or driving on a slope.

[0024] In one possible embodiment, the detection module is specifically used to: detect the deployment position of the image sensor based on the optical flow information; determine whether the deployment position of the image sensor is the same as the target deployment position, the target deployment position being the deployment position corresponding to the image sensor in a corresponding relationship, the corresponding relationship including the corresponding relationship between at least one image sensor and at least one deployment position; when the deployment position of the image sensor is the same as the target deployment position, determine that the deployment position of the image sensor matches the image sensor; or, when the deployment position of the image sensor is different from the target deployment position, determine that the deployment position of the image sensor does not match the image sensor.

[0025] In a possible implementation, the processing module is further configured to: update the corresponding relationship when the deployment position of the image sensor does not match the image sensor.

[0026] In a possible embodiment, it also includes: a communication module, used to send indication information to the deserializer, the indication information carries the updated correspondence relationship, and the indication information is used to indicate that the information of the second image frame sequence deserialized by the deserializer corresponds to the deployment position of the image sensor, and the second image frame sequence is collected by the image sensor.

[0027] In a possible embodiment, after the corresponding relationship is updated, the acquisition module is further used to: acquire a second image frame sequence, where the second image frame sequence is captured by the image sensor; the processing module is further used to: determine optical flow information of the second image frame sequence; and the detection module is further used to detect whether the deployment position of the image sensor matches the image sensor based on the optical flow information.

[0028] In one possible embodiment, the processing module is further used to: determine an update result based on a detection result of whether the deployment position of the image sensor matches the image sensor; if the deployment position of the image sensor matches the image sensor, the update result indicates that the update is successful; or, if the deployment position of the image sensor does not match the image sensor, the update result indicates that the update failed.

[0029] In a possible implementation, the communication module is further configured to push the update result.

[0030] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a processor and a memory, wherein the memory is used to store a computer program, and the processor is used to call and run the computer program stored in the memory to execute the method in the first aspect or each possible implementation method.

[0031] In a fourth aspect, an embodiment of the present application provides a vehicle, comprising: the control device in the second aspect or each possible implementation method.

[0032] In a fifth aspect, an embodiment of the present application provides a chip, comprising: a processor for calling and executing computer instructions from a memory, so that a device equipped with the chip executes the method in the first aspect or each possible implementation.

[0033] In a sixth aspect, an embodiment of the present application provides a computer-readable storage medium for storing computer program instructions, wherein the computer program enables a computer to execute the method in the first aspect or each possible implementation manner.

[0034] In a seventh aspect, an embodiment of the present application provides a computer program product, comprising computer program instructions, which enable a computer to execute the method in the first aspect or each possible implementation manner. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] FIG1 is a schematic diagram of the deployment position of an image sensor of a vehicle provided in an embodiment of the present application.

[0036] FIG2 is a schematic diagram of an image processing process provided in an embodiment of the present application.

[0037] FIG3 is a schematic structural diagram of an on-board computing platform of a vehicle provided in an embodiment of the present application.

[0038] FIG4 is a schematic structural diagram of an electronic device provided in an embodiment of the present application.

[0039] FIG5 a is a schematic diagram of an optical flow provided in an embodiment of the present application.

[0040] FIG5 b is a schematic diagram of another optical flow provided in an embodiment of the present application.

[0041] FIG6 is a flow chart of a method for detecting a sensor deployment position provided in an embodiment of the present application.

[0042] FIG7 is a schematic diagram of a sensor position detection method based on optical flow vanishing point according to an embodiment of the present application.

[0043] FIG8 is a schematic diagram of a sensor position detection method based on optical flow direction according to an embodiment of the present application.

[0044] FIG9 is a schematic diagram of a process for detecting sensor position based on optical flow information provided in an embodiment of the present application.

[0045] FIG10 is a flow chart of a method for detecting a sensor deployment position provided in an embodiment of the present application.

[0046] FIG11 a is a flow chart of a method for detecting a sensor deployment position provided in an embodiment of the present application.

[0047] FIG11 b is a flow chart of a method for detecting a sensor deployment position provided in an embodiment of the present application.

[0048] FIG12 is a schematic block diagram of a control device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0049] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.

[0050] The vehicle in the embodiments of the present application can be an intelligent vehicle, such as an autonomous driving vehicle that implements all functions as automatic control, or an assisted driving vehicle that implements some functions as automatic control to provide driving assistance, or it can be an ordinary vehicle. For the sake of convenience, it is collectively referred to as a vehicle below. When the vehicle in the embodiments of the present application is an ordinary vehicle, the vehicle can be automatically controlled by electronic devices outside the vehicle. The present application can implement vehicle control for any type of vehicle, such as fuel vehicles and new energy vehicles.

[0051] Intelligent driving uses artificial intelligence to assist or replace human driving, potentially addressing the shortcomings of human driving. Through extensive research and development, the performance and technology of the various sensors and computers required for intelligent driving have significantly improved, while costs are also gradually decreasing. As a result, intelligent driving has been widely applied in a variety of areas, including automated parking, adaptive cruise control (ACC) following, automatic emergency braking, lane departure warning, and autonomous driving.

[0052] The computing platform for intelligent driving services, or what can be called an onboard computing platform, such as a mobile data center (MDC), is deployed with software that runs various intelligent driving functions and serves as the "brain" of the intelligent driving system. Various sensors are connected to the onboard computing platform through different transmission methods. The algorithms running on the onboard computing platform process the signals and transmit them to downstream control units. Multiple image sensors connected to the onboard computing platform can capture images of the vehicle's surroundings, allowing the onboard computing platform to perform processing based on these images, such as sensor calibration and visual perception computing.

[0053] The image sensors deployed in the vehicle may include a front view sensor 001 , a left front view sensor 002 , a right front view sensor 003 , a rear view sensor 004 , a left rear view sensor 005 , and a right rear view sensor 006 as shown in FIG1 .

[0054] Among them, the forward-looking sensor 001 can be used to capture images directly in front of the vehicle, the left forward-looking sensor 002 is used to capture images on the left side of the vehicle at a certain angle to the vehicle's forward direction, and the right forward-looking sensor 003 is used to capture images on the right side of the vehicle at a certain angle to the vehicle's forward direction. The forward-looking sensor 001, or what can be called a front-facing sensor, can be deployed on the front side of the vehicle, such as on the front windshield or the front structure; the left forward-looking sensor 002 can be deployed on the vehicle body, such as on the left door or window of the vehicle; and the right forward-looking sensor 003 can be deployed on the vehicle body, such as on the right door or window of the vehicle.

[0055] Rearview sensor 004 can be used to capture images directly behind the vehicle. Left rearview sensor 005 is used to capture images at a certain angle to the left side of the vehicle's rearward direction. Right rearview sensor 006 is used to capture images at a certain angle to the right side of the vehicle's rearward direction. Rearview sensor 004 can be deployed on the rear side of the vehicle, such as on the rear windshield or rear structural components. Left rearview sensor 005 can be deployed in the left rearview mirror, and right rearview sensor 006 can be deployed in the right rearview mirror.

[0056] It should be understood that the number of image sensors and their deployment locations in the vehicle described above are merely examples and are not intended to be limiting. A vehicle may include more or fewer image sensors than those described above. For example, the number of forward-looking sensors 001 may be one or more. When there are multiple forward-looking sensors 001, the forward-looking sensors 001 may include different types of sensors. For example, the forward-looking sensors 001 may include a forward-looking telephoto sensor 011 and a forward-looking wide-angle sensor 012.

[0057] It should be understood that the on-board computing platform can perform sensor calibration or visual perception calculations based on images from multiple different "perspectives." Referring to Figure 2, image sensor A and image sensor B are image sensors deployed in the vehicle with different "perspectives." For example, according to a pre-set sensor mapping, image sensor A is the vehicle's forward-facing sensor, and image sensor B is the vehicle's rear-facing sensor. The images captured by image sensors A and B form serial data and are input into a deserializer. The deserializer deserializes the input data to obtain image A captured by image sensor A and image B captured by image sensor B. Image A is then used as the forward-facing image and image B as the rear-facing image, and the image signal processing (ISP) unit performs sensor calibration or visual perception calculations. However, if the image sensors are deployed in the wrong position and the images they capture have the wrong "perspective," for example, if image A captured by image sensor A in Figure 2 is used as the rear-facing image and image B captured by image sensor B is used as the forward-facing image, the on-board computing platform will still use image A as the forward-facing image and image B as the rear-facing image for sensor calibration or visual perception calculations, resulting in erroneous processing results.

[0058] Generally speaking, there should be at least two image sensors deployed in the wrong position. For example, a forward-facing sensor may be deployed in the position of a rear-facing sensor, and a rear-facing sensor may be deployed in the position of a forward-facing sensor. Another example is a forward-facing sensor may be deployed in the position of a rear-facing sensor, a rear-facing sensor may be deployed in the position of a left front-facing sensor, and a left front-facing sensor may be deployed in the position of a forward-facing sensor. Of course, this application is not limited to this; for example, one image sensor in a vehicle may be deployed in the wrong position.

[0059] It should be noted that this application does not limit the lens type of the image sensor, and it can be, for example, a normal lens, a fisheye lens, a telephoto lens, a wide-angle lens, etc. The image sensor in the embodiment of this application can also be replaced with other sensors in the vehicle, such as a lidar, an infrared sensor, etc.

[0060] In order to solve the above problems, the embodiment of the present application considers realizing sensor anti-reverse by software detection method. Figure 3 is a structural diagram of a vehicle-mounted computing platform provided by an embodiment of the present application. As shown in Figure 3, the vehicle-mounted computing platform 100 may include: an anti-reverse detection module 110, an anti-reverse processing module 120, a sensor calibration module 130, and an intelligent driving module 140. Among them, the anti-reverse detection module 110 can perform anti-reverse detection based on the image collected by the image sensor. It should be understood that anti-reverse detection is the detection of the deployment position of the image sensor; further, the anti-reverse detection module 110 can send the detection result to the anti-reverse processing module 120, and the anti-reverse processing module 120 can automatically adjust the image sensor deployed in the wrong position by adjusting the corresponding relationship (or mapping relationship) of the image sensor, such as the corresponding relationship between the image sensor and the deployment position; after the anti-reverse processing module 120 adjusts the image sensor deployed in the wrong position, the calibration module 130 can perform sensor calibration based on the images collected by multiple image sensors; the intelligent driving module 140 controls the vehicle for intelligent driving based on the images collected by multiple image sensors based on the calibration results.

[0061] It should be noted that the division of the units / modules in the above devices is only a division of logical functions, and in actual implementation, they can be fully or partially integrated into one physical entity, or they can be physically separated.

[0062] In some embodiments, the above-mentioned vehicle-mounted computing platform 100 can be implemented as an electronic device 200 as shown in FIG. 4 .

[0063] Figure 4 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. As shown in Figure 4, the electronic device 200 may include: a processor 210 and a memory 220. The processor 210 and the memory 220 communicate with each other through an internal connection path. Optionally, the memory 220 may include a read-only memory and a random access memory, and provide instructions and data to the processor 210. A portion of the memory 220 may also include a non-volatile random access memory. The memory 220 may be a separate device or integrated in the processor 210. The processor 210 may be used to execute instructions stored in the memory 220, and when the processor 210 executes instructions stored in the memory 220, the processor 210 is used to execute the various steps and / or processes of the method embodiment of the present application.

[0064] During implementation, each step of the method in the embodiment of the present application can be performed by a hardware processor in the processor 210, or by a combination of hardware and software modules in the processor 210. The software module can be located in a storage medium well-known in the art. The storage medium is located in the memory 220. The processor 210 reads the information in the memory 220 and, in conjunction with its hardware, completes the steps of the following method.

[0065] In some embodiments, the apparatus 200 may further include an input interface 230. The processor 210 may control the input interface 230 to communicate with other devices or chips, and specifically, may obtain information or data sent by other devices or chips.

[0066] In some embodiments, the apparatus 200 may further include an output interface 240. The processor 210 may control the output interface 240 to communicate with other devices or chips, and specifically, may output information or data to other devices or chips.

[0067] In some embodiments, the device 200 can implement the corresponding processes of each method in the embodiments of the present application. For the sake of brevity, they will not be repeated here.

[0068] It should be noted that the processor 210 in the embodiment of the present application can be an integrated circuit chip with signal processing capabilities. The above-mentioned processor 210 can be a microprocessor, a central processor unit (CPU), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component. The various methods, steps, and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed.

[0069] It is understood that the memory in the embodiments of the present application may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), and direct RAM bus RAM (DR RAM). It should be noted that the memory of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0070] To facilitate understanding, the terms involved in this application are first explained below.

[0071] 1. Image frame: refers to the smallest unit in the continuous image stream (or video) collected by the image sensor.

[0072] An image frame sequence may be composed of multiple frames of image data in a temporal sequence. This application does not limit the continuity between the image frames included in the image frame sequence. For example, a continuous segment of image frames captured by an image sensor may constitute an image frame sequence, or a sampling of image frames from a continuous segment of image frames captured by an image sensor may constitute an image frame sequence.

[0073] 2. Optical flow: It is a concept related to the detection of object motion in the field of view, and is used to describe the pixel motion of a spatially moving object on the imaging plane, such as the pixel motion of a feature point on the imaging plane in an image frame sequence. Feature points can be extracted from an image frame sequence, such as extracting corner points (such as Harris corner points, Shi-Tomasi corner points, etc.) from any image frame in the image frame sequence as feature points. By using an object tracking algorithm, such as sparse optical flow tracking, the position of the feature point on the next frame of the image can be obtained, and the feature points are repeatedly tracked. The coordinates of the feature points on the first frame to the last frame of the image are connected to form lines connecting the feature points, as shown in Figures 5a and 5b. Preferably, feature point lines that are too long or too short can be treated as noise and filtered out.

[0074] Optical flow methods are widely used in computer vision and other image processing fields. In the embodiments of the present application, optical flow information is used to represent information related to optical flow, such as at least one of the position of the optical flow vanishing point, the optical flow direction, or the optical flow speed.

[0075] Optical flow vanishing point: refers to the intersection of the lines connecting the feature points of the image frame sequence obtained using the optical flow method.

[0076] Optical flow direction: This refers to the vector direction of the lines connecting the feature points of an image frame sequence, obtained using the optical flow method. The direction of optical flow is generally described by the divergence or convergence of the lines connecting the feature points. For example, if the lines connecting multiple feature points in an image frame sequence diverge in the vector direction, the optical flow direction is outward divergence (see Figure 5b). If the lines connecting multiple feature points in an image frame sequence converge in the vector direction (see Figure 5a), the optical flow direction is inward convergence.

[0077] Optical flow speed: refers to the moving speed of the feature points in the image frame sequence obtained using the optical flow method.

[0078] To facilitate understanding of the embodiments of the present application, it should be noted that: "at least one" refers to one or more, and "more" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: the existence of A alone, the existence of A and B at the same time, and the existence of B alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b and c can represent: a, or b, or c, or a and b, or a and c, or b and c, or a, b and c. Wherein a, b and c can be single or multiple, respectively.

[0079] It should also be understood that in the embodiments of the present application, detecting whether the image sensor's deployment position matches the image sensor can also be expressed as detecting whether the image sensor's deployment position is incorrect. Detecting whether the image sensor's deployment position does not match the image sensor is equivalent to detecting whether the image sensor's deployment position is incorrect; detecting whether the image sensor's deployment position matches the image sensor is equivalent to detecting whether the image sensor's deployment position is correct. When these two expressions are used interchangeably, their meanings are consistent.

[0080] The execution subject of the embodiments of the present application may be a control device, which may be implemented as an electronic device, chip, chip system, or other functional module in the vehicle that can call and execute programs. The control device may be, for example, the above-mentioned vehicle-mounted computing platform, or may be equipped with the above-mentioned vehicle-mounted computing platform.

[0081] The following will describe in detail the method for detecting the sensor deployment position provided by the embodiment of the present application with reference to the accompanying drawings.

[0082] Figure 6 is a flow chart of a method 300 for detecting a sensor deployment location according to an embodiment of the present application. As shown in Figure 6 , the method 300 may include some or all of the following processes.

[0083] S310: Acquire a first image frame sequence, where the first image frame sequence is acquired by an image sensor deployed in a vehicle.

[0084] S320: Determine optical flow information of the first image frame sequence.

[0085] S330 , detecting whether the deployment position of the image sensor matches the image sensor based on the optical flow information.

[0086] As previously mentioned, a vehicle can be equipped with multiple different image sensors, such as image sensors with different "viewing angles," and detection can be used to determine if one or more image sensors are incorrectly positioned. For ease of understanding, the embodiments of this application use one of the multiple image sensors in a vehicle as an example. The solutions in the embodiments of this application are applicable to other image sensors in the vehicle. Of course, this application is also applicable to scenarios where a single image sensor is deployed in a vehicle.

[0087] In the above-mentioned S310, the control device may obtain a first image frame sequence from the image sensor, where the first image frame sequence is composed of multiple frames of images captured by the image sensor in a chronological order. The first image frame sequence may be composed of a continuous section of image frames captured by the image sensor, or may be composed of a number of image frames sampled from a continuous section of image frames captured by the image sensor. Optionally, the first image frame may be reported by the image sensor, or the first image frame may be read by the control device, or the first image frame may be forwarded by another device, and this application does not limit this.

[0088] In the above S320, the control device may determine the optical flow information of the first image frame sequence. Exemplarily, the optical flow information includes but is not limited to: at least one of the position information of the optical flow vanishing point, the optical flow direction, or the optical flow speed.

[0089] It should be noted that the first image frame sequence is named to distinguish it from the second image frame sequence mentioned below. "First" and "second" are not used to limit the scope of the embodiments of the present application. The first image frame sequence and the second image frame sequence mentioned below can both be referred to as image frame sequences.

[0090] Exemplarily, the control device may extract feature points (such as Harris corner points, Shi-Tomasi corner points, etc.) from the image frame a1, wherein the image frame a1 may be any frame in the first image frame sequence. Further, the control device obtains the position of the feature points on the next image frame (such as image frame a2) by using an object tracking algorithm, such as sparse optical flow tracking, and repeatedly tracks the feature points in the first image frame sequence by using the object tracking algorithm until the last image frame of the first image frame sequence (such as image frame a1) is obtained. n ), connect image frame a1 to image frame a n The coordinates of the feature points on the image are used to form a feature point line. Generally, the number of extracted feature points is multiple, and accordingly, multiple feature point lines are ultimately formed. Furthermore, the control device defines the intersection of the multiple feature point lines as the optical flow vanishing point. The position information of the optical flow vanishing point in the optical flow information is the position of the optical flow vanishing point in the imaging plane of the image sensor.

[0091] Exemplarily, the control device may determine the optical flow direction and / or optical flow velocity based on the direction of the lines connecting the multiple feature points in the first image frame sequence. Optionally, the optical flow direction and optical flow velocity may be obtained when the specific position of the optical flow vanishing point on the imaging plane is obtained.

[0092] When the image sensor is deployed at different locations in the vehicle, the optical flow information of the first image frame sequence captured by the image sensor exhibits different characteristics due to differences in the image capture perspective. Therefore, in S330 above, the control device may detect the deployment position of the image sensor that captured the first image frame sequence based on the optical flow information of the first image frame sequence.

[0093] Exemplarily, the control device can detect the deployment position of the image sensor based on the optical flow information, and determine whether the deployment position of the image sensor is the same as the target deployment position, where the target deployment position is the deployment position corresponding to the image sensor in the corresponding relationship. It should be understood that the corresponding relationship may include a correspondence between at least one image sensor and at least one deployment position. In other words, each image sensor has a fixed deployment position (or a preset deployment position) in the vehicle, and the corresponding relationship is used to indicate the target deployment position corresponding to each image sensor in the vehicle. Optionally, the deployment position corresponding to the image sensor in the corresponding relationship can be expressed by an identifier. Only when each image sensor is deployed at its corresponding target deployment position for image acquisition can the accuracy of sensor calibration and perception processing of the vehicle's machine vision system be ensured.

[0094] Furthermore, when the detected deployment position of the image sensor is the same as the target deployment position, the control device determines that the deployment position of the image sensor matches the image sensor, or determines that the deployment position of the image sensor is correct. When the detected deployment position of the image sensor is different from the target deployment position, the control device determines that the deployment position of the image sensor does not match the image sensor, or determines that the deployment position of the image sensor is incorrect.

[0095] The relationship between the deployment position of the image sensor and the optical flow information of the image frame sequence captured by the image sensor is exemplarily described below.

[0096] For example, as shown in FIG7 , the optical flow vanishing points of the image frame sequences captured by the forward and rearward sensors are located in the center region of the imaging plane; the optical flow vanishing points of the image frame sequences captured by the left forward and right rearward sensors are located in the right region of the imaging plane; and the optical flow vanishing points of the image frame sequences captured by the right forward and left rearward sensors are located in the left region of the imaging plane. The center region, right region, and left region can all be preset regions within the image coordinate system of the imaging plane.

[0097] For example, assuming that the image sensor that captures the first image frame sequence is a forward-looking sensor, and the optical flow vanishing point of the first image frame sequence is not in the middle area of ​​the imaging plane, the control device can determine that the deployment position of the image sensor is incorrect.

[0098] For example, as shown in Figure 8, the optical flow directions of the image frame sequence captured by the front-view sensor, the image frame sequence captured by the left front-view sensor, or the image frame sequence captured by the right front-view sensor all diverge outward; the optical flow directions of the image frame sequence captured by the rear-view sensor, the image frame sequence captured by the left rear-view sensor, and the image frame sequence captured by the right rear-view sensor all converge inward.

[0099] For example, assuming that the image sensor that captures the first image frame sequence is a forward-looking sensor, and the optical flow direction of the first image frame sequence converges inward, the control device determines that the deployment position of the image sensor is incorrect.

[0100] Exemplarily, the optical flow velocity of the image frame sequence captured by the forward-looking telephoto sensor is greater than a threshold, and the optical flow velocity of the image frame sequence captured by the forward-looking wide-angle sensor is less than the threshold. Optionally, depending on different application scenarios or different threshold settings, it can be stipulated that when the optical flow velocity of the image frame sequence is equal to the threshold, the sensor that captures the image frame sequence is the forward-looking telephoto sensor or the forward-looking wide-angle sensor.

[0101] For example, assuming that the image sensor that captures the first image frame sequence is a forward-looking telephoto sensor, and the optical flow velocity of the first image frame sequence is less than a threshold, the control device determines that the deployment position of the image sensor is incorrect.

[0102] Based on the correlation between the optical flow information of the image frame sequence and the deployment position of the image sensor that captured the image frame sequence, as some possible examples of the above S330, referring to FIG. 9 , the control device may detect the deployment position of the image sensor by combining the position of the optical flow vanishing point, the optical flow direction, and the optical flow speed, and may include the following steps:

[0103] S4, the control device determines whether the optical flow vanishing point of the first image frame sequence is located in the middle area of ​​the imaging plane of the image sensor.

[0104] If the control device determines that the optical flow vanishing point of the first image frame sequence is in the middle area of ​​the imaging plane of the image sensor, S41 is executed; if the control device determines that the optical flow vanishing point of the first image frame sequence is not in the middle area of ​​the imaging plane of the image sensor, S42 is executed.

[0105] S41 , the control device determines whether the direction of the optical flow of the first image frame sequence diverges outward.

[0106] If the control device determines that the optical flow direction of the first image frame sequence is outwardly divergent, S411 is executed; if the control device determines that the optical flow direction of the first image frame sequence is inwardly convergent, the control device determines that the image sensor is deployed at the deployment position of the rearview sensor.

[0107] S411 , the control device determines whether the optical flow speed of the first image frame sequence is greater than or equal to a threshold.

[0108] If the control device determines that the optical flow velocity of the first image frame sequence is greater than or equal to a threshold, the control device determines that the image sensor is deployed at the deployment position of the forward-looking telephoto sensor. If the control device determines that the optical flow velocity of the first image frame sequence is less than the threshold, the control device determines that the image sensor is deployed at the deployment position of the forward-looking wide-angle sensor.

[0109] S42: The control device determines whether the optical flow vanishing point of the first image frame sequence is located in the left area of ​​the imaging plane of the image sensor.

[0110] If the control device determines that the optical flow vanishing point of the first image frame sequence is in the left area of ​​the imaging plane of the image sensor, S421 is executed; if the control device determines that the optical flow vanishing point of the first image frame sequence is not in the left area of ​​the imaging plane of the image sensor, S422 is executed.

[0111] S421: The control device determines whether the image sensor is deployed at a deployment position of a right front view sensor or a deployment position of a left rear view sensor according to whether the optical flow direction of the first image frame sequence diverges outward.

[0112] In S421 above, if the control device determines that the direction of the optical flow of the first image frame sequence is outwardly divergent, the control device determines that the image sensor is deployed at the deployment position of the right front view sensor. If the control device determines that the direction of the optical flow of the first image frame sequence is not outwardly divergent (e.g., inwardly converging), the control device determines that the image sensor is deployed at the deployment position of the left rear view sensor.

[0113] S422: The control device determines whether the image sensor is deployed at a deployment position of a left front view sensor or a deployment position of a right rear view sensor according to whether the optical flow direction of the first image frame sequence diverges outward.

[0114] In S422 above, if the control device determines that the direction of the optical flow of the first image frame sequence is outwardly divergent, the control device determines that the image sensor is deployed at the deployment position of the left front view sensor. If the control device determines that the direction of the optical flow of the first image frame sequence is not outwardly divergent (e.g., inwardly converging), the control device determines that the image sensor is deployed at the deployment position of the right rear view sensor.

[0115] It should be noted that the embodiment shown in FIG9 is merely exemplary. When the control device detects the deployment position of the image sensor that captured the first image frame sequence based on at least two of the position of the optical flow vanishing point, the optical flow direction, or the optical flow velocity of the first image frame sequence, this application also defines the execution order of each judgment condition. When the control device detects the deployment position of the image sensor that captured the first image frame sequence based on the optical flow information of the first image frame sequence, if other relevant judgment conditions or execution orders are adopted, they still fall within the scope of protection of this application.

[0116] Therefore, in an embodiment of the present application, the control device detects whether the deployment position of the image sensor matches the image sensor based on the optical flow information of the first image frame sequence collected by the image sensor, and realizes anti-reflection of the image sensor in the vehicle through software detection. The deployment position of the image sensor that collects the first image frame sequence is detected based on the optical flow information of the first image frame sequence, thereby improving the accuracy of anti-reflection detection.

[0117] In some embodiments, to improve the accuracy of detection results, the control device may obtain vehicle driving information before detecting the image sensor's deployment position based on optical flow information, and detect the image sensor's deployment position in combination with the vehicle's driving information. Exemplarily, the control device may obtain vehicle driving information, which may include but is not limited to the vehicle's driving speed and / or driving status, where the vehicle's driving status includes but is not limited to at least one of: driving, turning, or driving on a sloped road.

[0118] In one possible implementation, when the vehicle's driving information meets a preset condition, the control device detects whether the image sensor's deployment position matches the image sensor. For example, when the vehicle's driving speed meets a speed threshold and the vehicle is traveling straight ahead, the control device may detect whether the image sensor's deployment position matches the image sensor based on optical flow information.

[0119] Optionally, the vehicle's driving information may also include chassis information and / or positioning information (e.g., information collected by the vehicle's positioning system). The vehicle control device may determine whether the vehicle is traveling on an open road based on the collected chassis information and / or positioning information. When the vehicle is traveling on an open road, the control device detects the deployment position of the image sensor. The term "open road" is used relative to indoor roads (e.g., parking lots) and can be any road section, including, for example, urban roads and highways.

[0120] It should be noted that, in order to ensure that the vehicle's driving information meets the preset conditions when the image sensor captures the first image frame sequence, the control device may execute the above S310 when the vehicle's driving information meets the preset conditions; in some scenarios, considering the processing delay, the control device may execute the above S320 or S330 when it is determined that the vehicle's driving information meets the preset conditions.

[0121] Referring to Figure 10, the control device can read the image frame sequence and driving information collected by the image sensor in real time; the control device determines whether the preset conditions are met through the driving information; if the driving information does not meet the preset conditions, the control device can continue to read the image frame sequence and driving information collected by the image sensor until the driving information meets the preset conditions, and the control device obtains the corresponding first image frame sequence, and performs feature extraction and optical flow tracking on the first image frame sequence to obtain optical flow information of the first image frame.

[0122] In another implementation, the control device can combine driving information to detect whether the deployment position of the image sensor matches the image sensor. In this implementation, the impact of the vehicle's driving information on the optical flow information of the image sensor can be considered. For example, when the vehicle is driving faster, the optical flow speed is faster, so the threshold value of the optical flow speed used to distinguish between the forward telephoto sensor and the forward wide-angle sensor is larger. For example, when the vehicle is turning or driving on a slope, the optical flow direction and optical flow vanishing point of the first image frame sequence will be affected. When detecting the deployment position of the image sensor based on the optical flow direction and / or the position of the optical flow vanishing point, it is necessary to analyze it in conjunction with the vehicle's driving status.

[0123] To improve processing efficiency and reduce time and labor costs, in some embodiments, the control device may update the aforementioned correspondence, or in other words, update the mapping relationship of the image sensors in the vehicle, upon detecting an incorrect deployment position of an image sensor. In one understanding, updating the mapping relationship of the image sensors in the vehicle may involve updating the "perspective" type of the image sensors, such as updating a front-view sensor to a rear-view sensor. For example, if image sensor A is a front-view sensor and the control device determines that image sensor A is deployed at the deployment position of a rear-view sensor, the control device may update the deployment position of image sensor A in the correspondence relationship to the deployment position of the rear-view sensor.

[0124] Exemplarily, the control device may send an indication message to the deserializer, the indication message carrying an updated correspondence relationship, the indication message being used to indicate that the information of the second image frame sequence deserialized by the deserializer corresponds to the deployment position of the image sensor, wherein the second image frame sequence is acquired by the image sensor that acquired the first image frame sequence, and the second image frame sequence may be acquired by the image sensor after the mapping relationship of the image sensor is updated. As shown in FIG2 , image sensor A deployed at the rearview sensor position, before the correspondence relationship is updated, image sensor A is a frontview sensor, and after being deserialized by the deserializer, the second image frame sequence acquired by image sensor A is considered to be an image frame sequence acquired by the frontview sensor. After the correspondence relationship is updated, image sensor A is a rearview sensor, and after being deserialized by the deserializer, the second image frame sequence acquired by image sensor A is considered to be an image frame sequence acquired by the rearview sensor.

[0125] To ensure that the updated correspondence is accurate, the control device may obtain a second image frame sequence and determine optical flow information of the second image frame sequence, and then detect whether the deployment position of the image sensor matches the image sensor based on the optical flow information of the second image frame sequence.

[0126] For example, the control device may determine the update result based on a detection result of whether the deployment position of the image sensor matches the image sensor. For example, if it is determined that the deployment position of the image sensor matches the image sensor, the update result indicates that the update is successful; for another example, if the deployment position of the image sensor does not match the image sensor, the update result indicates that the update failed.

[0127] Exemplarily, the control device may push the update result, for example, present the update result through a display screen, or send the update result to other devices.

[0128] In some embodiments, to ensure stable operation of the system, a time limit can be reserved for the control device to update the corresponding relationship. As shown in FIG11a , the control device can determine, based on the first image frame sequence, whether the deployment position of the image sensor that captured the first image frame sequence is incorrect; if the deployment position of the image sensor is determined to be correct, the control device can report an update result (e.g., update success or deployment correct); if the deployment position of the image sensor is determined to be incorrect, the control device can update the corresponding relationship and run a timer (e.g., 8 seconds, 12 seconds, etc.), and then report the update result (e.g., update result indicating update success or update failure).

[0129] Based on the above example, the control device can determine whether the deployment position of the image sensor is incorrect according to the detection period. When the detection period is short, the control device can still periodically determine whether the deployment position of the image sensor is incorrect during the process of updating the correspondence. In this case, if the control device determines that the deployment position of the image sensor is incorrect, it can determine whether the correspondence update has been executed. If the correspondence update has been executed, the control device will repeat the operations of determining whether the deployment position of the image sensor is incorrect and determining whether the correspondence update has been executed before the timer expires. After the indication timer expires, the update result (such as the update result indicating that the update is successful or failed) is reported, see Figure 11b.

[0130] In some embodiments, the control device may initiate detection of the sensor deployment location in response to a human-computer interaction operation by the user, and push detection results and / or update results to the user.

[0131] FIG12 is a schematic block diagram of a control device according to an embodiment of the present application. As shown in FIG12 , the control device 500 includes: an acquisition module 510 , a processing module 520 , and a detection module 530 .

[0132] Among them, the acquisition module 510 can be used to acquire a first image frame sequence, which is collected by an image sensor deployed in the vehicle; the processing module 520 can be used to determine the optical flow information of the first image frame sequence; and the detection module 530 can be used to detect whether the deployment position of the image sensor matches the image sensor based on the optical flow information.

[0133] In some embodiments, the optical flow information includes the position of the optical flow vanishing point in the imaging plane of the image sensor; the optical flow vanishing point is in the middle area of ​​the imaging plane of the image sensor, and the image sensor is deployed at the position of the front-view sensor or the rear-view sensor; or, the optical flow vanishing point is in the left area of ​​the imaging plane of the image sensor, and the image sensor is deployed at the position of the left-rear side-view sensor or the right-front side-view sensor; or, the optical flow vanishing point is in the right area of ​​the imaging plane of the image sensor, and the image sensor is deployed at the position of the left-front side-view sensor or the right-rear side-view sensor.

[0134] In some embodiments, the optical flow information includes the direction of the optical flow; the direction of the optical flow diverges outward, and the image sensor is deployed at the position of the front-view sensor, the left front-view sensor, or the right front-view sensor; or, the direction of the optical flow converges inward, and the image sensor is deployed at the position of the rear-view sensor, the left rear-view sensor, or the right rear-view sensor.

[0135] In some embodiments, the optical flow information includes optical flow speed; if the optical flow speed is greater than or equal to a threshold, the image sensor is deployed at the position of the forward-looking telephoto sensor; or, if the optical flow speed is less than the threshold, the image sensor is deployed at the position of the forward-looking wide-angle sensor.

[0136] In some embodiments, the acquisition module 510 is also used to obtain driving information of the vehicle; the processing module 520 is also used to determine the optical flow information of the first image frame sequence based on the first image frame sequence when the driving information of the vehicle meets the preset conditions; wherein the driving information includes the driving speed and / or driving status of the vehicle, and the driving status includes at least one of straight driving, turning or driving on a slope.

[0137] In some embodiments, the detection module 530 is specifically used to: detect the deployment position of the image sensor based on the optical flow information; determine whether the deployment position of the image sensor is the same as the target deployment position, the target deployment position is the deployment position corresponding to the image sensor in the corresponding relationship, and the corresponding relationship includes a corresponding relationship between at least one image sensor and at least one deployment position; when the deployment position of the image sensor is the same as the target deployment position, determine that the deployment position of the image sensor matches the image sensor; or, when the deployment position of the image sensor is different from the target deployment position, determine that the deployment position of the image sensor does not match the image sensor.

[0138] In some embodiments, the processing module 520 is further configured to update the corresponding relationship when the deployment position of the image sensor does not match the image sensor.

[0139] In some embodiments, the control device 500 also includes: a communication module 530, which is used to send indication information to the deserializer, the indication information carries the updated correspondence relationship, and the indication information is used to indicate that the information of the second image frame sequence deserialized by the deserializer corresponds to the deployment position of the image sensor, and the second image frame sequence is collected by the image sensor.

[0140] In some embodiments, after updating the correspondence, the acquisition module 510 is further used to: acquire a second image frame sequence, where the second image frame sequence is captured by the image sensor; the processing module 520 is further used to: determine the optical flow information of the second image frame sequence; and the detection module 530 is further used to detect whether the deployment position of the image sensor matches the image sensor based on the optical flow information.

[0141] In some embodiments, the processing module 520 is further used to: determine an update result based on a detection result of whether the deployment position of the image sensor matches the image sensor; if the deployment position of the image sensor matches the image sensor, the update result indicates that the update is successful; or, if the deployment position of the image sensor does not match the image sensor, the update result indicates that the update failed.

[0142] In some embodiments, the communication module 540 is further used to push update results.

[0143] It should be understood that the specific process of each module executing the above corresponding steps has been described in detail in the above method embodiment, and for the sake of brevity, it will not be repeated here.

[0144] The division of the modules in the above apparatus 500 is merely a division of logical functions, and in actual implementation, they may be fully or partially integrated into one physical entity, or they may be physically separated.

[0145] An embodiment of the present application also provides a computer-readable storage medium for storing a computer program.

[0146] In some embodiments, the computer program enables the computer to execute the corresponding processes in the various methods of the embodiments of the present application. For the sake of brevity, they are not described here in detail.

[0147] An embodiment of the present application also provides a computer program product, including computer program instructions.

[0148] In some embodiments, the computer program instructions enable the computer to execute the corresponding processes in the various methods of the embodiments of the present application. For the sake of brevity, they are not repeated here.

[0149] The embodiment of the present application also provides a computer program.

[0150] In some embodiments, when the computer program runs on a computer, it enables the computer to execute the corresponding processes in the various methods of the embodiments of the present application. For the sake of brevity, they are not described here in detail.

[0151] An embodiment of the present application also provides a vehicle.

[0152] In some embodiments, the vehicle includes a control device or chip for the vehicle in the embodiments of the present application.

[0153] In some embodiments, the vehicle further includes the image sensor of the embodiments of the present application.

[0154] In some embodiments, the vehicle includes the deserializer of the embodiments of the present application.

[0155] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0156] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. A method for detecting a sensor deployment position, characterized in that: include: Acquire a first image frame sequence, where the first image frame sequence is acquired by an image sensor deployed in a vehicle; determining optical flow information of the first image frame sequence; According to the optical flow information, it is detected whether the deployment position of the image sensor matches the image sensor.

2. The method according to claim 1, characterized in that The optical flow information includes the position of the optical flow vanishing point on the imaging plane of the image sensor; The light flow vanishing point is in the middle area of ​​the imaging plane of the image sensor, and the image sensor is deployed at the position of the front-view sensor or the rear-view sensor; or, The light flow vanishing point is in the left area of ​​the imaging plane of the image sensor, and the image sensor is deployed at the position of the left rear side view sensor or the right front side view sensor; or, The optical flow vanishing point is in the right area of ​​the imaging plane of the image sensor, and the image sensor is deployed at the position of the left front side view sensor or the right rear side view sensor.

3. The method according to claim 1 or 2, characterized in that The optical flow information includes the optical flow direction; The direction of the optical flow diverges outwards, and the image sensor is deployed at the position of the front view sensor, the left front view sensor, or the right front view sensor; or, The optical flow direction converges inward, and the image sensor is deployed at the position of the rearview sensor, the left rearview sensor or the right rearview sensor.

4. The method according to any one of claims 1 to 3, characterized in that The optical flow information includes optical flow speed; The optical flow speed is greater than or equal to a threshold, and the image sensor is deployed at the position of a forward-looking telephoto sensor; or The optical flow speed is less than a threshold, and the image sensor is deployed at the position of the forward-looking wide-angle sensor.

5. The method according to any one of claims 1 to 4, characterized in that Also includes: Obtaining driving information of the vehicle; When the driving information of the vehicle meets a preset condition, determining optical flow information of the first image frame sequence according to the first image frame sequence; The driving information includes the driving speed and / or driving status of the vehicle, and the driving status includes at least one of straight driving, turning, or driving on a sloped road.

6. The method according to any one of claims 1 to 5, characterized in that The detecting, based on the optical flow information, whether the deployment position of the image sensor matches the image sensor includes: detecting a deployment position of the image sensor according to the optical flow information; Determining whether the deployment position of the image sensor is the same as a target deployment position, the target deployment position being the deployment position corresponding to the image sensor in a corresponding relationship, the corresponding relationship including a corresponding relationship between at least one image sensor and at least one deployment position; When the deployment position of the image sensor is the same as the target deployment position, determining that the deployment position of the image sensor matches the image sensor; or, When the deployment position of the image sensor is different from the target deployment position, it is determined that the deployment position of the image sensor does not match the image sensor.

7. The method according to claim 6, characterized in that Also includes: When the deployment position of the image sensor does not match the image sensor, the corresponding relationship is updated.

8. The method according to claim 7, characterized in that Also includes: Sending indication information to the deserializer, where the indication information carries the updated correspondence, where the indication information is used to indicate that information of the second image frame sequence deserialized by the deserializer corresponds to the deployment position of the image sensor, and the second image frame sequence is acquired by the image sensor.

9. The method according to claim 7 or 8, characterized in that After updating the corresponding relationship, the method further includes: Acquire a second image frame sequence, where the second image frame sequence is acquired by the image sensor; determining optical flow information of the second image frame sequence; According to the optical flow information, it is detected whether the deployment position of the image sensor matches the image sensor.

10. The method according to claim 9, characterized in that Also includes: determining an update result according to a detection result of whether the deployment position of the image sensor matches the image sensor; If the deployment position of the image sensor matches the image sensor, the update result indicates that the update is successful; or, If the deployment position of the image sensor does not match the image sensor, the update result indicates that the update has failed.

11. The method according to claim 10, characterized in that Also includes: Push the update result.

12. A control device, characterized in that: include: An acquisition module is configured to acquire a first image frame sequence, where the first image frame sequence is acquired by an image sensor deployed in the vehicle; a processing module, configured to determine optical flow information of the first image frame sequence; A detection module is used to detect whether the deployment position of the image sensor matches the image sensor according to the optical flow information.

13. The device according to claim 12, characterized in that The optical flow information includes the position of the optical flow vanishing point on the imaging plane of the image sensor; The light flow vanishing point is in the middle area of ​​the imaging plane of the image sensor, and the image sensor is deployed at the position of the front-view sensor or the rear-view sensor; or, The light flow vanishing point is in the left area of ​​the imaging plane of the image sensor, and the image sensor is deployed at the position of the left rear side view sensor or the right front side view sensor; or, The optical flow vanishing point is in the right area of ​​the imaging plane of the image sensor, and the image sensor is deployed at the position of the left front side view sensor or the right rear side view sensor.

14. The device according to claim 12 or 13, characterized in that The optical flow information includes the optical flow direction; The direction of the optical flow diverges outwards, and the image sensor is deployed at the position of the front view sensor, the left front view sensor, or the right front view sensor; or, The optical flow direction converges inward, and the image sensor is deployed at the position of the rearview sensor, the left rearview sensor or the right rearview sensor.

15. The device according to any one of claims 12 to 14, characterized in that The optical flow information includes optical flow speed; The optical flow speed is greater than or equal to a threshold, and the image sensor is deployed at the position of a forward-looking telephoto sensor; or The optical flow speed is less than a threshold, and the image sensor is deployed at the position of the forward-looking wide-angle sensor.

16. The device according to any one of claims 12 to 15, characterized in that The acquisition module is further used to acquire the driving information of the vehicle; The processing module is further configured to determine optical flow information of the first image frame sequence according to the first image frame sequence when the driving information of the vehicle meets a preset condition; The driving information includes the driving speed and / or driving status of the vehicle, and the driving status includes at least one of straight driving, turning, or driving on a sloped road.

17. The device according to any one of claims 12 to 16, characterized in that The detection module is specifically used for: detecting a deployment position of the image sensor according to the optical flow information; Determining whether the deployment position of the image sensor is the same as a target deployment position, the target deployment position being the deployment position corresponding to the image sensor in a corresponding relationship, the corresponding relationship including a corresponding relationship between at least one image sensor and at least one deployment position; When the deployment position of the image sensor is the same as the target deployment position, determining that the deployment position of the image sensor matches the image sensor; or, When the deployment position of the image sensor is different from the target deployment position, it is determined that the deployment position of the image sensor does not match the image sensor.

18. The device according to claim 17, characterized in that The processing module is further configured to: When the deployment position of the image sensor does not match the image sensor, the corresponding relationship is updated.

19. The device according to claim 18, characterized in that Also includes: A communication module is used to send indication information to the deserializer, where the indication information carries the updated correspondence relationship, and the indication information is used to indicate that the information of the second image frame sequence deserialized by the deserializer corresponds to the deployment position of the image sensor, and the second image frame sequence is acquired by the image sensor.

20. The device according to claim 18 or 19, characterized in that After the update correspondence, The acquisition module is further configured to: acquire a second image frame sequence, where the second image frame sequence is acquired by the image sensor; The processing module is further configured to: determine optical flow information of the second image frame sequence; The detection module is further configured to detect, based on the optical flow information, whether the deployment position of the image sensor matches the image sensor.

21. The device according to claim 20, characterized in that The processing module is further configured to: determining an update result according to a detection result of whether the deployment position of the image sensor matches the image sensor; If the deployment position of the image sensor matches the image sensor, the update result indicates that the update is successful; or, If the deployment position of the image sensor does not match the image sensor, the update result indicates that the update has failed.

22. The device according to claim 21, characterized in that Also includes: The communication module is used to push the update result.

23. An electronic device, characterized in that: include: A processor and a memory, the memory being used to store a computer program, the processor being used to call and run the computer program stored in the memory to execute the method according to any one of claims 1 to 11.

24. A vehicle, characterized in that: A control device comprising the control device according to any one of claims 12 to 22.

25. A chip, characterized in that: include: A processor, configured to call and execute computer instructions from a memory, so that a device equipped with the chip executes the method according to any one of claims 1 to 11.

26. A computer-readable storage medium, characterized in that Used to store computer program instructions, wherein the computer program causes a computer to execute the method according to any one of claims 1 to 11.

27. A computer program product, characterized in that The method comprises computer program instructions, which cause a computer to execute the method according to any one of claims 1 to 11.

Images