A control instruction generation method and device for visual sensor, and a control method and device

Abstract

The present application relates to the field of vehicle automatic driving, and specifically to a control instruction generation method for a visual sensor, wherein the visual sensor collects image data by scanning, and the method comprises: acquiring image data; determining an object that will collide with the vehicle according to the image data; generating an image acquisition range adjustment instruction according to the object; and sending the image acquisition range adjustment instruction to the visual sensor, wherein the image acquisition range adjustment instruction is used to instruct the visual sensor that originally collects images according to a preset range to collect images according to a temporary range, wherein the temporary range is smaller than the preset range and includes the area where the object is located. In an emergency, the vehicle can promptly obtain image data of an object that is identified as being about to collide with the vehicle, thereby reducing the occurrence of traffic accidents.

Description

Technical Field

[0001] The present application relates to the field of vehicle autonomous driving, and specifically to a control instruction generation method and device, and a control method and device for a visual sensor. Background Art

[0002] Sensors play an extremely important role in the autonomous driving platform. A number of different sensors can be installed on the vehicle, for example, visual sensors for the outside of the vehicle that can provide reversing images, front view images, rear view images, overhead images, and panoramic parking images; sensors for the cabin that can monitor whether passengers are driving fatigued, and sensors that can monitor the status of the instrument panel; and sensors for the Advanced Driving Assistance System (ADAS) that can provide forward collision warnings, lane departure warnings, automatic high beam control, traffic signal recognition, pedestrian detection, adaptive cruise control, blind spot detection, and night vision.

[0003] The autonomous driving control platform can be connected to a variety of sensors. After the data received by the sensor is transmitted to the autonomous driving control platform, the autonomous driving control platform performs a series of data processing and finally outputs control instructions for the vehicle. Under this premise, end-to-end low latency from the sensor receiving data to the vehicle executing the autonomous driving control instructions has become a goal that the industry continues to pursue. Summary of the invention

[0004] In view of the above problems in the prior art, the purpose of this application is to provide a control instruction generation method and device for a visual sensor, and a control method and device, which can reduce the time from when the visual sensor receives data to when the vehicle executes the automatic driving control instruction, and reduce the occurrence of traffic accidents caused by untimely acquisition of data of objects determined to be about to collide with the vehicle.

[0005] A first aspect of an embodiment of the present application provides a control instruction generation method for a visual sensor, wherein the visual sensor collects image data by scanning, and the method includes: acquiring image data; determining an object that will collide with a vehicle based on the image data; generating an image acquisition range adjustment instruction based on the object; and sending the image acquisition range adjustment instruction to the visual sensor, wherein the image acquisition range adjustment instruction is used to instruct the visual sensor, which originally collects images based on a preset range, to collect images based on a temporary range, wherein the temporary range is smaller than the preset range and includes an area where the object is located.

[0006] Through the above-mentioned settings, the visual sensor can immediately scan the area where the object that is identified as being about to collide with the vehicle is located in an emergency situation. There is no need to wait for the visual sensor to scan the preset range in the current cycle and then rescan the preset range, thereby reducing the size of the image acquisition range and the time to obtain image data of the object that is about to collide with the vehicle, thereby reducing the risk of traffic accidents caused by untimely image data acquisition of the object that is about to collide with the vehicle in an emergency situation.

[0007] In a possible implementation manner, the temporary range is a rectangle.

[0008] Through the above settings, the image acquisition range can be narrowed to the rectangular area where the object identified as being about to collide with the vehicle is located, thereby reducing the image acquisition time and preventing the situation in which risk avoidance is delayed due to long image acquisition time in an emergency.

[0009] In a possible implementation manner, the visual sensor includes a camera.

[0010] A second aspect of an embodiment of the present application provides a control method for a visual sensor, wherein the visual sensor collects image data by scanning, and the method comprises: obtaining an image collection range adjustment instruction; controlling the visual sensor to adjust the image collection range according to the image collection range adjustment instruction, wherein the image collection range adjustment instruction is used to instruct the visual sensor, which originally collects images according to a preset range, to collect images according to a temporary range, wherein the temporary range is smaller than the preset range and includes an area where an object identified as being about to collide with the vehicle is located.

[0011] Through the above-mentioned settings, the visual sensor can immediately obtain image data of the object identified as being about to collide with the vehicle, without waiting for the visual sensor to finish scanning the preset range in the current cycle and then rescan the entire preset range, thereby reducing the size of the image acquisition range, and reducing the time for obtaining image data of the object identified as being about to collide with the vehicle, thereby reducing the risk of traffic accidents caused by untimely acquisition of image data of the object identified as being about to collide with the vehicle in an emergency situation.

[0012] In a possible implementation manner, the temporary range is a rectangle.

[0013] Through the above settings, the image acquisition range can be narrowed to the rectangular area where the object identified as being about to collide with the vehicle is located, thereby reducing the image acquisition time and preventing the situation in which risk avoidance is not timely due to long image acquisition time in an emergency.

[0014] In a possible implementation manner, the temporary range of the current image acquisition cycle is adjusted, and when the acquired image range includes the temporary range, image acquisition is performed on the union of the acquired image range and the temporary range.

[0015] Through the above-mentioned setting, repeated image acquisition of the temporary range caused by full or partial duplication of the acquired image range and the temporary range is avoided, and the time spent on image acquisition is further reduced.

[0016] In a possible implementation manner, the visual sensor includes a camera.

[0017] A third aspect of an embodiment of the present application provides a control instruction generating device for a visual sensor, wherein the visual sensor collects image data by scanning, and the device includes: an image data acquisition module, used to acquire image data; an identification module, used to determine an object that will collide with a vehicle based on the image data; a control instruction generating module, used to generate an image acquisition range adjustment instruction based on the object; and a control instruction sending module, used to send the image acquisition range adjustment instruction to the visual sensor, wherein the image acquisition range adjustment instruction is used to instruct the visual sensor, which originally collects images according to a preset range, to collect images according to a temporary range, wherein the temporary range is smaller than the preset range and includes the area where the object is located.

[0018] In a possible implementation manner, the temporary range is a rectangle.

[0019] In a possible implementation manner, the visual sensor includes a camera.

[0020] A fourth aspect of an embodiment of the present application provides a control device for a visual sensor, wherein the visual sensor collects image data by scanning, and the device includes: a control instruction receiving module, which is used to obtain an image acquisition range adjustment instruction; a control module, which is used to control the visual sensor to adjust the image acquisition range according to the image acquisition range adjustment instruction, and the image acquisition range adjustment instruction is used to instruct the visual sensor that originally collects images according to a preset range to collect images according to a temporary range, wherein the temporary range is smaller than the preset range and includes an area where an object identified as being about to collide with the vehicle is located.

[0021] In a possible implementation manner, the temporary range is a rectangle.

[0022] In a possible implementation, the method further includes adjusting the temporary range of the current image acquisition cycle, and when the acquired image range includes the temporary range, performing image acquisition on a union of the acquired image range and the temporary range.

[0023] In a possible implementation manner, the visual sensor includes a camera.

[0024] A fifth aspect of an embodiment of the present application provides a driving risk prediction method, comprising:

[0025] Acquiring image data, wherein the image data is obtained by scanning with a visual sensor;

[0026] determining, based on the image data, an object that will collide with the vehicle;

[0027] generating an image acquisition range adjustment instruction according to the object;

[0028] sending the image acquisition range adjustment instruction to the visual sensor,

[0029] Obtain an image acquisition range adjustment instruction, and control the visual sensor to adjust the image acquisition range according to the image acquisition range adjustment instruction, wherein the image acquisition range adjustment instruction is used to instruct the visual sensor that originally acquires images according to a preset range to acquire images according to a temporary range, and the temporary range is smaller than the preset range and includes the area where the object is located.

[0030] In a possible implementation manner, the temporary range is a rectangle.

[0031] In a possible implementation, when the image acquisition range adjustment instruction is received, the temporary range of the current image acquisition cycle is adjusted, and when the acquired image range includes the temporary range, image acquisition is performed on the union of the acquired image range and the temporary range.

[0032] In a possible implementation manner, the visual sensor includes a camera.

[0033] The sixth aspect of the present application provides a computer-readable storage medium having program instructions stored thereon, which, when executed by a computer, enables the computer to execute any of the methods provided in the first, second and fifth aspects and their possible implementations.

[0034] The seventh aspect of the present application provides a computer program, and by running the program, a computer can execute any of the methods provided by the above-mentioned first, second and fifth aspects and their possible implementations, or function as any of the devices provided by the second aspect and its possible implementations.

[0035] These and other aspects of the present application will become more apparent from the following description of the embodiment(s). BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The following further illustrates the various features of the present application and the connections between the various features with reference to the accompanying drawings. The accompanying drawings are all exemplary, and some features are not shown in actual proportion. Some drawings may omit features that are conventional in the field involved in the present application and are not necessary for the present application, or additional features that are not necessary for the present application may be shown. The combination of the various features shown in the accompanying drawings is not intended to limit the present application. In addition, throughout this specification, the same figure numerals refer to the same content. The specific description of the drawings is as follows:

[0037] Figure 1a is a schematic diagram of the structure of a driving hazard prediction system provided in an embodiment of the present application;

[0038] Figure 1b is a schematic diagram of the structure of a control device for a visual sensor provided in an embodiment of the present application;

[0039] Figure 2a is a flow chart of a visual sensor control method provided by an embodiment of the present application;

[0040] Figure 2b is a sub-flow chart of the visual sensor control method provided in an embodiment of the present application;

[0041] Figure 3a-3f is an exemplary image of a traffic scene captured by a visual sensor provided in an embodiment of the present application;

[0042] Figure 4a-4d is a schematic diagram of a visual sensor provided in an embodiment of the present application scanning a scene according to different image acquisition range adjustment instructions;

[0043] Figure 5a-5d The visual sensor provided in the embodiment of the present application is 1 -t 4 An image of a scene captured at the moment.

[0044] Description of Reference Numerals

[0045] Visual sensor 10; visual sensor control device 11; ADAS computing and control device 20; perception module 21; control module 22; dangerous area identification module 23; control instruction generation module 24; ISP30; ADAS platform 100; visual sensor scanning control module 110; comparison module 111; driving hazard prediction system 200; ECU210; image 300; intersection 301; road 302; traffic sign 303; traffic light 304; vehicles 305, 306, 307, 308, 401; pedestrians 309, 501. DETAILED DESCRIPTION

[0046] The technical solutions involved in the specific implementation of the present application are described below in conjunction with the drawings in the implementation. Before describing the specific content of the technical solution, the terms used in the present application are briefly explained.

[0047] The words "first, second, third, etc." or module A, module B, module C and other similar terms in the specification and claims are only used to distinguish similar objects and do not represent a specific ordering of the objects. It is understandable that the specific order or sequence can be interchanged where permitted so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0048] The term "comprising" as used in the description and claims should not be interpreted as being limited to what is listed thereafter; it does not exclude other elements or steps. Therefore, it should be interpreted as specifying the presence of the features, integers, steps or components mentioned, but does not exclude the presence or addition of one or more other features, integers, steps or components and groups thereof. Therefore, the expression "a device comprising means A and B" should not be limited to a device consisting of components A and B only.

[0049] References to "one embodiment" or "an embodiment" in this specification mean that a particular feature, structure, or characteristic described in conjunction with the embodiment is included in at least one embodiment of the present application. Therefore, the phrases "in one embodiment" or "in an embodiment" appearing in various places in this specification do not necessarily refer to the same embodiment, but may refer to the same embodiment. In addition, in one or more embodiments, the particular features, structures, or characteristics can be combined in any appropriate manner, as would be apparent to one of ordinary skill in the art from this disclosure.

[0050] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the present application. In case of any inconsistency, the meaning described in this specification or the meaning derived from the contents recorded in this specification shall prevail. In addition, the terms used herein are only for the purpose of describing the embodiments of the present application and are not intended to limit the present application.

[0051] Advanced Driving Assistance System (ADAS) includes multiple sensors and data processing platforms. Its working principle is to collect data about the moving object and its surrounding environment through multiple sensors installed on the moving object. After the data is processed and analyzed by the data processing platform, the moving object's driving path is planned and control commands are sent to the control module to perform related operations.

[0052] Image Signal Processor (ISP) is a device used to process the image output by the front-end image sensor.

[0053] The Electronic Control Unit (ECU) is used to calculate various input data and process various input instructions according to a pre-designed program, and further control various actuators to perform various predetermined control functions.

[0054] First, the deficiencies in the prior art discovered by the inventor are described:

[0055] In the current ADAS system, the visual sensor scans the image multiple times per frame. After each scan, the visual sensor can obtain information about a line of images from the real world and then transmit it to the ISP. After the ISP receives a complete frame of images consisting of multiple lines of images, it processes the image. The processed image is transmitted to the data processing module in the ADAS data processing platform for algorithm processing, and finally generates control instructions.

[0056] When the frame rate of the visual sensor is 30fps (30 frames per second), the exposure interval between two adjacent frames is about 33ms. As a result, the ADAS data processing platform cannot obtain image information in the real world within the 33ms time interval. If the data processing platform recognizes a danger signal and needs to immediately obtain the image of the specified area in the current lens screen, it will take up to 33ms to wait. In scenarios such as emergency avoidance and emergency decision-making, there is a large delay problem.

[0057] In view of such problems in the prior art, the embodiments of the present application provide a control instruction generation method and device for a visual sensor, a control method and device, and a driving hazard prediction method and system. The image of a specified position in the current lens image can be obtained in emergency avoidance and emergency decision-making scenarios, thereby suppressing traffic accidents caused by delayed image acquisition.

[0058] An overview of one embodiment of the present application is as follows.

[0059] First embodiment: Driving risk prediction system.

[0060] Figure 1a The block structure of a driving risk prediction system 200 including a vision sensor control device 11 and an ADAS calculation and control device 20 is shown.

[0061] The driving hazard prediction system 200 of this embodiment is located on a vehicle. The vehicle is, for example, a car. The driving hazard prediction system 200 may include a plurality of sensors, a plurality of sensor control devices, an ADAS platform 100, and a vehicle ECU 210. The visual sensor control device 11 is connected to the visual sensor 10, and is used to control the visual sensor 10 to collect image data.

[0062] The ADAS platform 100 can be connected to the visual sensor 10 and the visual sensor control device 11 through a line, and is used to send instructions to the visual sensor control device 11 and receive and process data collected by the visual sensor and other sensors. The vehicle ECU 210 can receive control instructions from the ADAS platform 100 and further control the vehicle to perform corresponding driving operations.

[0063] In some embodiments of the present application, the sensor may be a visual sensor 10, such as a camera, or a sensor such as a laser radar sensor, a millimeter wave sensor, etc. that can obtain data by scanning. Correspondingly, the sensor control device may be a visual sensor control device 11. The ADAS platform 100 may include: a driving hazard prediction device configured as an ADAS computing and control device 20, and an ISP30. The ADAS computing and control device 20 may include: a perception module 21, a control module 22, a dangerous area identification module 23, and a control instruction generation module 24. The vehicle ECU210 connection can be connected to the ADAS computing and control device 20 in a wired or wireless manner, and is used to further control the vehicle to perform corresponding operations according to the instructions generated by the ADAS computing and control device 20.

[0064] The visual sensor 10 is used to obtain image data of traffic scenes, and can be installed at different positions of the vehicle. The visual sensor 10 has a pixel array composed of a plurality of unit pixels arranged in two dimensions. When obtaining image data of a scene, the visual sensor can scan the preset range line by line in a time sequence, and each time a row of images in an area is scanned, the image data of a row of areas can be obtained, and the data from the row of images can be output to the ISP30 through a line. When the last row of images is scanned, the visual sensor adds an end mark to the image data of the last row of scans, indicating that a cycle of image acquisition is completed. The ISP30 starts to process the received image data of the preset range according to the end mark. The visual sensor 10 can scan the temporary range indicated for image acquisition according to the image acquisition range adjustment instruction. When the visual sensor 10 finishes scanning the last row of images in the temporary range, the visual sensor 10 adds an end mark to the last row of image data, indicating that the image acquisition of the current cycle is completed. The ISP30 starts to process the received temporary image according to the end mark.

[0065] ISP30 is a processor that performs image processing on the image data output from the visual sensor 10. ISP30 receives the row image data output from the visual sensor 10 in chronological order, and after receiving the row image data with an end mark, performs gamma correction, color interpolation processing, and automatic white balance processing on all the row image data output from the visual sensor 10. ISP30 can be a chip integrated in the ADAS platform 100, or a chip integrated with the visual sensor 10. In this embodiment, ISP30 is integrated in the ADAS platform 100, and is provided with an image interface to receive the image data sent by the visual sensor 10 through a data line.

[0066] The ADAS computing and control device 20 is used to process image data obtained from multiple sensors and generate control instructions to control the vehicle ECU to perform corresponding operations. The ADAS computing and control device 20 has a perception module 21, a control module 22, a dangerous area identification module 23 and a control instruction generation module 24.

[0067] The perception module 21 is a device capable of performing algorithmic processing on the image data from the ISP 30. It is used to perform image detection on the acquired image to identify the object in the image and obtain the information of the object. The object may be a traffic participant in the surrounding environment of the vehicle. The traffic participants may include pedestrians, surrounding vehicles, traffic signs, and obstacles. The information of the object may include: the position of the object in the world coordinate system, the size of the object. The perception module can use a neural network model, an object recognition algorithm, a Structure from Motion (SFM) algorithm, video tracking, and other computer vision technologies for image detection.

[0068] The perception module 21 determines the position and size of the traffic participant according to the pixel coordinates of the identified traffic participant and the calibration parameters of the visual sensor 10. The calibration parameters can be the internal parameters, external parameters, position information, etc. of the visual sensor lens. The perception module 21 obtains the position information corresponding to any pixel coordinate point of the traffic participant in the world coordinate system according to the pixel coordinates of the same traffic participant in the first image and the second image obtained by the visual sensor at different positions of the vehicle at the same time, the internal parameters and external parameters corresponding to the visual sensor, and then determines the position of the traffic participant; according to the image area composed of multiple pixel coordinates and the scaling factor of the visual sensor, the size of the traffic participant in the world coordinate system is determined. During the operation of the vehicle, the perception module 21 can obtain the positioning information of the vehicle in real time through the inertial navigation device / laser radar, or can use satellite positioning technology (for example: GPS technology) to obtain the positioning information of the vehicle in real time, or can use any other existing positioning technology to obtain the positioning information of the vehicle in real time, and the embodiment of the present application is not limited to this. The positioning information of the vehicle may include longitude and latitude, altitude, and posture information of the vehicle (such as the direction of the front of the vehicle). The latitude and longitude and altitude in the positioning information of the above-mentioned vehicle are data in the world coordinate system (also known as the geographic coordinate system). The distance of the traffic participant relative to the vehicle is determined based on the positioning information of the vehicle and the position of the traffic participant. The perception module 21 can also receive the driving trajectory of the vehicle currently planned by the control module 22. The perception module 21 includes: an input interface, an output interface, a program memory, a working memory and a microcontroller. The input interface is used to receive image data output from the ISP; the output interface is used to output the information of the traffic participant to the control module 22 and the dangerous area identification module 23; the microcontroller can read commands from the program memory and execute each process in sequence. The microcontroller temporarily expands the program pre-stored in the program memory into the working memory and performs various actions according to its command group. The algorithm for obtaining the information of the traffic participant in the scene data is implemented by a combination of a microcontroller and software. The software can be a module constituting a computer program for performing specific processing corresponding to each functional block. Such a computer program can be stored in a program memory.

[0069] The danger zone identification module 23 is an algorithm processing device different from the perception module 21, which can determine whether the traffic participant is about to collide with the vehicle based on the information of the traffic participant obtained from the perception module 21. The judgment conditions for whether the traffic participant is about to collide with the vehicle may include: at the current moment, the position of the same traffic participant intersects with the current driving trajectory of the vehicle, the distance of the same traffic participant relative to the vehicle is less than the first distance, and the size of the same traffic participant exceeds a preset value. When the traffic participant meets the above judgment criteria, it indicates that the traffic participant is about to collide with the vehicle, and it is necessary to immediately obtain the image of the traffic participant to further plan the vehicle's driving trajectory. It is not possible to wait for the visual sensor to scan the preset range of the current traffic scene in chronological order and then re-scan according to the preset range.

[0070] The control instruction generation module 24 is used to generate an image acquisition range adjustment instruction when the traffic participant is judged to be about to collide with the vehicle, and control the visual sensor that originally collects image data according to the preset range to collect images according to a temporary range, which is smaller than the preset range and includes the area where the traffic participant identified as about to collide with the vehicle is located. The image acquisition range adjustment instruction is used to instruct the visual sensor to collect images of the area where the traffic participant identified as about to collide with the vehicle is located. According to different scanning modes of the visual sensor, the temporary range can be the row area or column area where the traffic participant is located for image collection, and the temporary range can also be the rectangular, triangular, or circular area where the traffic participant is located. For example, when the traffic participant is a pedestrian, the temporary range is a rectangular area including the area where the pedestrian is located. The control instruction sending module is used to send the image acquisition range adjustment instruction to the visual sensor 10. The control instruction sending module can be a data line, one end of which is connected to the control instruction generation module, and the other end of which is connected to the visual sensor control device through the ISP30. The image acquisition range adjustment instruction can also be sent to the visual sensor control device 11 in the form of signal transmission.

[0071] The control module 22 is a computing device different from the perception module 21 and the dangerous area identification module 23 , and is used to plan the vehicle's driving path and generate vehicle driving control instructions based on the information of traffic participants obtained by the perception module 21 .

[0072] The vehicle ECU 210 includes a microprocessor (CPU), a memory (ROM, RAM), an input / output interface (I / O), an analog-to-digital converter (A / D), and an integrated circuit. The vehicle ECU is connected to the control module 23 through the input interface via a connecting line, and is used to receive the vehicle driving control instructions generated by the control module, and further control each actuator to perform the corresponding driving operation according to the vehicle driving control instructions.

[0073] The visual sensor control device 11 can control the visual sensor 10 to scan images according to a temporary range based on the image acquisition range adjustment instruction output from the control instruction generation module 24. The temporary range is obtained by reducing the preset range, and the temporary range includes traffic participants identified by the dangerous area recognition module 23 as being about to collide with the vehicle.

[0074] Figure 1b The module schematic diagram of the visual sensor control device 11 is shown. The visual sensor control device 11 may include a visual sensor scanning control module 110 and a comparison module 111 .

[0075] The visual sensor scanning control module 110 may be a control circuit for controlling the visual sensor 10, and is integrated on the visual sensor 10, and is used to control the scanning range of the visual sensor according to the image acquisition range adjustment instruction. For example, when the visual sensor scanning control module 110 receives an image acquisition range adjustment instruction for data acquisition from the Nth row to the Mth row, the visual sensor 10 is controlled to complete the row area currently scanned, and start scanning from the Nth row until the Mth row is scanned.

[0076] The comparison module 111 can adjust the temporary range of the current image acquisition cycle according to the image acquisition range adjustment instruction of the control instruction generation module 24. The comparison module 111 compares the temporary range with the unacquired range where image acquisition has not been performed and the acquired range where image acquisition has been performed. When the acquired range includes the temporary range, the comparison module 111 generates a final adjustment instruction for performing image acquisition on the union of the acquired range and the temporary range.

[0077] When the captured range does not include the temporary range, that is, the temporary range is within the uncaptured range, the comparison module 111 generates a final adjustment instruction for image capture of the temporary range, and sends the final adjustment instruction to the visual sensor scanning control module 110. The visual sensor completes the scan of the current row area, adds an end mark to the image data corresponding to the current row area, and then starts to capture images of the temporary range. For example, when the visual sensor scans the Ath row, the comparison module 111 receives an image capture range adjustment instruction for image capture from the Nth row to the Mth row (M>N>A). The visual sensor scanning control module 110 controls the visual sensor 11 to stop the current scan, and starts scanning from the Nth row until the scan of the Mth row is completed.

[0078] When the already captured range completely encompasses the temporary range, there is no need to perform image capture on the temporary range again. Only the image data of the already captured range needs to be sent to the ISP. The comparison module 111 generates a final adjustment instruction to end the image capture of the currently already captured range and sends this final adjustment instruction to the vision sensor scanning control module 110. The vision sensor scanning control module 110 controls the vision sensor to perform image capture again according to the preset range. For example, when the vision sensor is performing image capture on row A, when the comparison module 111 receives an image capture range adjustment instruction to perform image capture on rows N to M (A > M > N), the already captured image range already includes the temporary range. Only the currently already captured image range needs to be sent to the ISP for processing. Therefore, the vision sensor scanning control module 110 controls the vision sensor 11 to complete the scanning of row A and adds an end mark to row A, ending the image capture of the currently already captured range.

[0079] When the already captured range partially encompasses the temporary range, there is no need to re-perform image capture on the temporary range again. Only continue to perform image capture on the basis of the currently captured image range. The comparison module 111 generates a final adjustment instruction to continue image capture and sends this final adjustment instruction to the vision sensor scanning control module 110. The vision sensor scanning control module 110 receives the image capture range adjustment instruction and controls the vision sensor 10 to continue image capture. For example, when the vision sensor has scanned from row 1 to row P, when the comparison module 111 receives an image capture range adjustment instruction to perform image capture on rows N to M (N < P < M), the already captured range already partially includes the temporary range. Only the vision sensor needs to continue scanning until the temporary range is scanned completely. Therefore, the vision sensor scanning control module 110 controls the vision sensor 11 to continue to execute the current scanning sequence until the scanning of the area of row M is completed.

[0080] The operation modes of the various components of the driving risk prediction system will be described below with reference to the accompanying drawings.

[0081] The scanning mode of the vision sensor.

[0082] Figure 3a-3f An exemplary image 300 of a traffic scene captured by a vision sensor with a screen resolution of 1920×1080 is shown. This image 300 is a color image presented in black and white lines to comply with the provisions of the Implementing Regulations of the Patent Law. The image 300 includes a road 302 with an intersection 301, traffic signs 303, traffic lights 304, other vehicles 305, 306, 307, 308, and pedestrians 309.

[0083] When the visual sensor control device 11 does not receive the image acquisition range adjustment instruction from the control instruction generation module 24, the visual sensor 10 adjusts the image acquisition range according to the preset range. Figure 3a Scan the scene shown. Figure 3a The image range shown is scanned line by line starting from line L1 until line L1080 is scanned to complete the scanning of one frame of image. In order to clearly show the vehicles and pedestrians in the figure, only line L1, line L2 and line L1080 are schematically drawn with dotted lines.

[0084] When the visual sensor control device 11 receives the image acquisition range adjustment instruction from the control instruction generation module 24, the scene is scanned according to the image acquisition range adjustment instruction. Different image acquisition range adjustment instructions have different scanning modes of the visual sensor. The following introduces the scanning modes of the visual sensor according to three different image acquisition range adjustment instructions.

[0085] First scanning mode: the image acquisition range adjustment instruction indicates scanning a rectangular area.

[0086] like Figure 3b As shown, when the visual sensor control device 11 receives the image acquisition range adjustment instruction for scanning the L33-L44 rows when the visual sensor scans the L22th row, the comparison module 111 compares the L33-L44 row area indicated by the image acquisition range adjustment instruction with the scanned L1-L22 row area and the unscanned L23-L1080 row area. The scanned image range does not include the area indicated by the image acquisition range adjustment instruction to scan, and the comparison module 111 generates the final adjustment instruction for scanning the L33-L44 rows and sends it to the visual sensor scanning control module 110. The visual sensor scanning control module 110 controls the visual sensor to stop scanning the L22th row, and starts scanning from the L33th row line by line until scanning to the L44th row. When the L44th row is scanned, the scanning starts again from the L1st row.

[0087] like Figure 3c As shown, when the visual sensor control device receives the image acquisition range adjustment instruction for scanning the L14-L44 rows when the visual sensor scans the L100th row, the comparison module 111 compares the scanned L1-L100th row area with the L14-L44th row area indicated by the image acquisition range adjustment instruction. The scanned image range portion completely includes the range indicated by the image acquisition range adjustment instruction to scan, and the comparison module 111 generates a final adjustment instruction to end the scan and sends it to the visual sensor scanning control module 110. The visual sensor scanning control module 110 controls the visual sensor 10 to add an end mark to the image data of the L100th row, and then starts scanning from the L1th row.

[0088] like Figure 3d As shown, when the visual sensor scanning control module receives the image acquisition range adjustment instruction for scanning the L14-L500 rows when the visual sensor scans the L200th row, the comparison module 111 compares the scanned area of ​​the L1-L200 rows with the area for scanning the L14-L500 rows indicated by the image acquisition range adjustment instruction. The scanned image range portion includes the area indicated by the image acquisition range adjustment instruction to scan, and the comparison module 111 generates a final adjustment instruction to continue scanning until all the areas indicated by the image acquisition range adjustment instruction to scan are scanned, and sends it to the visual sensor scanning control module 110. The visual sensor scanning control module 110 controls the visual sensor 10 to continue scanning until scanning reaches the L500th row. When the L500th row is scanned, scanning starts again from the L1th row.

[0089] After scanning the area specified by the image acquisition range adjustment instruction, the visual sensor 10 is not limited to restarting scanning from the L1th row, but can also continue scanning the area specified by the image acquisition range adjustment instruction line by line, or can start scanning from any row as needed.

[0090] The second scanning mode: the image acquisition range adjustment instruction specifies scanning of the elliptical area where the traffic participants are located.

[0091] like Figure 3e As shown, the image acquisition range adjustment instruction indicates to scan the elliptical area X in the image range of the visual sensor. In the following description, for the same processing in the second scanning mode as the first scanning mode, the content of the first embodiment is cited for description, or only briefly described.

[0092] When the visual sensor scanning control module 110 does not receive the image acquisition range adjustment instruction from the control instruction generation module 24, the visual sensor 10 scans the scene according to the preset range. This is the same as in the first scanning mode and will not be described in detail.

[0093] like Figure 3e As shown, when the visual sensor scanning control module 110 receives a signal to scan area X when the visual sensor 10 scans the L22th row, it controls the visual sensor to complete the scanning of the L22th row and then starts scanning area X. After the scanning of area X is completed, it starts scanning again from the L1th row.

[0094] The third scanning mode: the image acquisition range adjustment instruction specifies scanning of the rectangular area where the traffic participants are located.

[0095] like Figure 3fAs shown, the image acquisition range adjustment instruction is to scan the area Y. In the following description, for the same processing in the third scanning mode as that in the first scanning mode, the content of the first embodiment is cited for description, or only a brief description is given.

[0096] When the visual sensor scanning control module does not receive the image acquisition range adjustment instruction from the control instruction generation module, the visual sensor scans the scene according to the preset range. This is the same as in the first scanning mode and will not be described in detail here.

[0097] When the visual sensor scanning control module 110 receives the image acquisition range adjustment instruction from the sensor control instruction generation module 24 when the visual sensor 10 scans the L22th row, it controls the visual sensor to adjust the scanning area. When the image acquisition range adjustment instruction is to scan the area Y, the visual sensor 10 is controlled to scan the area Y after completing the scanning of the L22th row, and then start scanning the area Y, until the scanning of the area Y is completed, and then start scanning again from the L1th row.

[0098] Alternatively, the image acquisition range adjustment instruction may specify scanning of a circular, square, or other shaped area where the traffic participants are located. Alternatively, the instruction may specify scanning of a pixel area constituting the traffic participants, as long as the area can completely contain the area where the traffic participants identified as those about to collide with the vehicle are located.

[0099] Combine the following Figure 3a The scene shown illustrates the working modes of the perception module 21 , the danger zone identification module 23 , the control instruction generation module 24 and the control module 22 .

[0100] Refer again Figure 3a An image of the scene shown. After receiving the first image and the second image obtained by the visual sensors located at different positions of the vehicle at the same time, the perception module 21 uses a neural network model, an object recognition algorithm, a structure-in-motion algorithm, video tracking and other computer vision technologies to detect the image, extract features in the image, and match them with preset features to determine traffic participants such as traffic signs 303, traffic lights 304, pedestrians 309 and surrounding vehicles 305, 306, 307, 308 in the image; determine the position and size of the traffic participants based on the pixel coordinates of each identified traffic participant and the calibration parameters of the visual sensor 10; and determine the distance of the traffic participant relative to the vehicle based on the acquired positioning information of the vehicle.

[0101] After the perception module 21 obtains information about the positions of multiple traffic participants, the sizes of traffic participants, the distances of traffic participants relative to the vehicle, and the driving trajectory of the vehicle currently planned by the control module 22, the information of these traffic participants is sent to the danger zone identification module 23. The danger zone identification module 23 determines whether the traffic participant is about to collide with the vehicle based on the information of these traffic participants. The criteria for determining whether the traffic participant is about to collide with the vehicle may include: at the current moment, the position of the same traffic participant intersects with the current driving trajectory of the vehicle, the distance of the same traffic participant relative to the vehicle is less than the first distance, and the size of the same traffic participant exceeds a preset value. When the above adjustments are met, it indicates that the image of the traffic participant needs to be acquired immediately, and it is impossible to wait for the visual sensor to scan the preset range in chronological order.

[0102] The control instruction generation module 24 generates an image acquisition range adjustment instruction for scanning the area where one or more traffic participants are located based on the traffic participants that are about to collide with the vehicle identified by the dangerous area recognition module 23, and sends the image acquisition range adjustment instruction to the sensor scanning control module 11 through ISP30.

[0103] The following is an explanation of the working modes of the danger zone identification module 23 and the control instruction generation module 24 in combination with different scenarios with reference to the accompanying drawings:

[0104] Scenario 1: The vehicle ahead is approaching the vehicle

[0105] Figure 4a-4d The values ​​of 1 -t 4 The 4 frames of images captured by the visual sensor at the moment. Figure 4a-4d The change in the position of the front vehicle 401 in the image range can be seen in FIG. Among them, the proportion of the pixel connection area of ​​the front vehicle 401 in the entire image range of the visual sensor gradually increases, and the distance between the front vehicle and the vehicle decreases. The dangerous area recognition module 23 4 At the moment, the position of the front vehicle 401 intersects the current driving track of the vehicle, the distance of the front vehicle relative to the vehicle is less than the first distance, and the size of the front vehicle exceeds the preset value. The front vehicle 401 is determined to be a traffic participant that will collide with the vehicle. The control instruction generation module 24 generates an image acquisition range adjustment instruction for acquiring images of the area where the front vehicle 401 is located according to the recognition result of the dangerous area recognition module 23. Figure 4dAs shown, the area where the front vehicle 401 is located corresponds to the area from rows L500 to L900 in the image range of the visual sensor, and the control instruction generation module generates an image acquisition range adjustment instruction for acquiring images of the area from rows L500 to L900, and sends the image acquisition range adjustment instruction to the sensor scanning control module through the ISP.

[0106] Scenario 2: Pedestrians approach the vehicle

[0107] Figure 5a-5d The values ​​of 5 -t 8 The 4 frames of images captured by the visual sensor at the moment. Figure 5a-5d It can be seen from FIG. 5 that the position of pedestrian 501 in the image range of the visual sensor changes. Figure 5a-5d It can be seen from FIG. 5 that the pedestrian 501 in front moves from the right side of the image range to the center of the image range. The dangerous area recognition module 23 detects the pedestrian 501 in front at t 8 The current driving track of the vehicle at the position at the time intersects, the distance of the front pedestrian 501 relative to the vehicle is less than the first distance, and the size of the front pedestrian exceeds the preset value, and the front pedestrian 501 is determined to be a traffic participant that will collide with the vehicle. The control instruction generation module 24 generates an image acquisition range adjustment instruction for the area where the front pedestrian 501 is located. Figure 5d As shown, the area where the pedestrian 501 in front is located corresponds to the L460th to L980th rows in the image range of the visual sensor. Therefore, the control instruction generation module 24 generates an image acquisition range adjustment instruction for image acquisition of the area from the L460th to the L980th rows, and sends the image acquisition range adjustment instruction to the sensor scanning control module through the ISP.

[0108] Refer again Figure 3a After the perception module 21 obtains the traffic participant information of multiple traffic participants such as traffic signs 303, traffic lights 304, pedestrians 309, and surrounding vehicles 305, 306, 307, 308 in multiple frames of scene images, the traffic participant information of these traffic participants is also sent to the control module 24. The control module plans the vehicle's driving path and generates control instructions based on the traffic participant information obtained by the perception module, and the vehicle ECU further controls various components of the vehicle to perform corresponding operations based on the control instructions.

[0109] Second embodiment: Driving risk prediction method

[0110] Figure 2a-2b A flow chart of a driving risk prediction method is shown.

[0111] The driving risk prediction method includes the following steps:

[0112] Step S1: The visual sensor scans the scene.

[0113] The visual sensor scans the preset image range of the visual sensor line by line in time sequence, and after each line of the area is scanned, the image data of the line of the area is transmitted to the ISP.

[0114] Step S2: ISP processes the received image.

[0115] The ISP is configured to process all previously received images of the row regions after receiving the image of the row region with the end mark.

[0116] ISP's image processing may include adjustment of image parameters such as white balance and scanning, as well as optimization of image noise.

[0117] Step S3: Use the perception module to perform algorithm processing on the image data processed by the ISP to obtain information about the traffic participants in the image.

[0118] Among them, the description of the algorithm processing can refer to the description of the perception module in the first embodiment of the present application.

[0119] Step S41: The danger zone identification module determines an object that will collide with the vehicle.

[0120] The danger zone identification module determines whether the traffic participant is about to collide with the vehicle based on the acquired traffic participant information. When the following conditions are met, it is determined that the traffic participant is likely to collide with the vehicle and the traffic participant is the object that is about to collide with the vehicle: at the current moment, the position of the same traffic participant intersects with the current driving trajectory of the vehicle, the distance of the same traffic participant relative to the vehicle is less than the first distance, and the size of the same traffic participant exceeds a preset value.

[0121] Step S42: When the traffic participant is determined to be about to collide with the vehicle, the control instruction generation module generates an image acquisition range adjustment instruction to perform image acquisition on a temporary range including the traffic participant.

[0122] The temporary range may be a rectangular area including the traffic participant.

[0123] Step S43: The visual sensor control device receives and executes the image acquisition range adjustment instruction generated in step S42.

[0124] Wherein, step S43 may further include the following sub-steps:

[0125] Step S431: The comparison module receives the image acquisition range adjustment instruction generated by the control instruction generation module, and compares the temporary range with the scanned image range and the unscanned image range respectively.

[0126] The comparison module adjusts the temporary range of the current image acquisition cycle according to the comparison result, and when the acquired image range includes the temporary range, performs image acquisition on the union of the acquired image range and the temporary range.

[0127] Step S4321: When the acquired image range includes the entire temporary range, the comparison module generates a final adjustment instruction to end the current image acquisition.

[0128] Step S4331: after completing image acquisition of the current row area, the visual sensor scanning control module 110 controls the visual sensor to add an end mark to the current row area, thereby completing image acquisition of the current acquired image range.

[0129] Step S4322: When the acquired image range includes part of the temporary range, the comparison module generates a final adjustment instruction to continue image acquisition.

[0130] Step S4332: the visual sensor scanning control module 110 controls the visual sensor to complete the currently captured row area, and continues to capture images of the remaining temporary ranges that have not been captured.

[0131] Step S4323: When the captured image range does not include the temporary range, the comparison module generates a final adjustment instruction for performing image capture on the temporary range.

[0132] Step S4333: The visual sensor scanning control module 110 controls the visual sensor to complete the currently acquired row area and starts to acquire images of the temporary range.

[0133] Among them, the description of step S43 and its sub-steps executed by the visual sensor control device can refer to the description of the visual sensor control device in the first embodiment of the present application. For the sake of brevity, only a brief description is given here.

[0134] The generated final adjustment instruction or image acquisition range adjustment instruction can be transmitted to the visual sensor scanning control module through the ISP. The visual sensor scanning control module controls the visual sensor to scan according to the control instruction.

[0135] While executing steps S41-S43, step S5 is also executed: the control module plans the vehicle's driving path according to the acquired traffic participant information and generates driving control instructions.

[0136] Step S6: The vehicle ECU controls the vehicle to perform corresponding operations according to the driving control instructions.

[0137] After step S43 is executed, step S2, step S3, step S5 and step S6 may be executed in sequence to avoid collision between the vehicle and traffic participants.

[0138] Combine the following Figure 4a-4d The scenario shown illustrates a specific embodiment of the driving hazard prediction method.

[0139] Step S100: The visual sensor detects Figure 4a-4d The image range shown is scanned.

[0140] The visual sensor transmits the image data to the ISP after scanning each row of the area. When the last row of the area is scanned, the visual sensor adds an end mark to the last row of the area.

[0141] Step S200: After receiving the row region with the end mark, the ISP processes the images of all the row regions received previously and sends the processed images to the perception module.

[0142] Step S300: The perception module performs algorithm processing on the image processed by the ISP to obtain Figure 4a-4d The image shown includes information about vehicle 401.

[0143] Step S410: the danger zone identification module determines whether the vehicle 401 is about to collide with the host vehicle based on the acquired information of the vehicle 401.

[0144] from Figure 4a-4d It can be seen that at t 1 -t 4 During this period, the proportion of vehicle 401 in the entire image range of the visual sensor gradually increases. 4 At the moment, the position of vehicle 401 intersects the current driving track of the host vehicle, the distance of vehicle 401 relative to the host vehicle is less than the first distance, and the size of vehicle 401 exceeds the preset value. Vehicle 401 is about to collide with the host vehicle, and the position information of vehicle 401 needs to be obtained immediately.

[0145] Step S420: Generate an image acquisition range adjustment instruction for scanning the vehicle 401.

[0146] The image acquisition range adjustment instruction is to scan the L508-L806 row area where the vehicle 401 is located.

[0147] Step S430: the visual sensor control device receives and executes the image acquisition range adjustment instruction.

[0148] Step S110: The visual sensor scans the area of ​​rows L508-L806 where the vehicle 401 is located.

[0149] Step S210: The ISP processes the received image of the L508-L806 row area and sends the processed image to the perception module.

[0150] Step S310: The perception module performs algorithmic processing on the image of the L508-L806 row area to obtain information about the vehicle 401 in the image of the L508-L806 row area, which may include the position of the vehicle 401, the distance of the vehicle 401 relative to the vehicle, and the size of the vehicle 401.

[0151] Step S500: The control module plans the driving path of the vehicle according to the information of vehicle 401 and generates control instructions.

[0152] Step S600: The vehicle ECU controls the vehicle to perform corresponding operations according to the control instructions.

[0153] Third Embodiment: Computer Readable Storage Medium

[0154] An embodiment of the present application also provides a computer-readable storage medium on which a computer program is stored. The program is executed by a processing device to perform a braking method and calculation. The method includes at least one of the solutions described in the above embodiments.

[0155] The computer storage medium of the embodiment of the present application can adopt any combination of one or more computer-readable media. Computer-readable media can be computer-readable signal media or computer-readable storage media. Computer-readable storage media can be, for example, but not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or devices, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: electrical connections with one or more wires, portable computer disks, hard disks, random access storage devices (RAM), read-only storage devices (ROM), erasable programmable read-only storage devices (EPROM or flash memory), optical fibers, portable compact disk read-only storage devices (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above. In this document, computer-readable storage media can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it.

[0156] A computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, device, or device component.

[0157] The program code embodied on the computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0158] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or service device. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect through the Internet).

[0159] Fourth embodiment: computer program

[0160] The fifth embodiment of the present application provides a computer program, and a computer can execute the control method provided in the embodiment of the present application or function as the above-mentioned control device by running the program.

[0161] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0162] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0163] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0164] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0165] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0166] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a service device, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory device (ROM), a random access memory device (RAM), a magnetic disk or an optical disk.

[0167] Note that the above are only preferred embodiments of the present application and the technical principles used. Those skilled in the art will understand that the present application is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the protection scope of the present application. Therefore, although the present application is described in more detail through the above embodiments, the present application is not limited to the above embodiments, and may also include more other equivalent embodiments without departing from the concept of the present application, all of which belong to the protection scope of the present application.

Claims

1. A control method for a visual sensor, It is characterized in that The visual sensor collects image data by scanning, and the method includes: Get image acquisition range adjustment instructions; Control the visual sensor to adjust the image acquisition range according to the image acquisition range adjustment instruction, The image acquisition range adjustment instruction is used to instruct the visual sensor that originally acquires images according to the preset range to acquire images according to a temporary range, wherein the temporary range is smaller than the preset range and includes an area where an object that is identified as about to collide with the vehicle is located, wherein: The step of controlling the visual sensor to adjust the image acquisition range according to the image acquisition range adjustment instruction includes: Comparing the temporary range with the acquired image range; Determine a final adjustment instruction according to the comparison result, wherein when the acquired image range completely includes the temporary range, the final adjustment instruction instructs to end image acquisition; when the acquired image range includes part of the temporary range, the final adjustment instruction instructs to continue image acquisition for the acquired image range until the acquisition of the temporary range is completed; when the acquired image range does not include the temporary range, the final adjustment instruction instructs to perform image acquisition for the temporary range; According to the final adjustment instruction, the visual sensor is controlled to adjust the image acquisition range.

2. The method according to claim 1, It is characterized in that The temporary range is a rectangle.

3. The control method for a visual sensor according to claim 1 or 2, It is characterized in that The visual sensor includes a camera.

4. A control device for a visual sensor, It is characterized in that The visual sensor collects image data by scanning, and the device includes: a control instruction receiving module, which is used to obtain an image acquisition range adjustment instruction, wherein the image acquisition range adjustment instruction is used to instruct the visual sensor that originally acquires images according to a preset range to acquire images according to a temporary range, wherein the temporary range is smaller than the preset range and includes an area where an object that is identified as about to collide with the vehicle is located; A comparison module, used for comparing the temporary range with the range of the image that has been collected according to the image collection range adjustment instruction; The comparison module is further used to determine a final adjustment instruction according to the comparison result, wherein when the acquired image range completely includes the temporary range, the final adjustment instruction instructs to end image acquisition; when the acquired image range includes part of the temporary range, the final adjustment instruction instructs to continue image acquisition for the acquired image range until the acquisition of the temporary range is completed; when the acquired image range does not include the temporary range, the final adjustment instruction instructs to perform image acquisition for the temporary range; A control module is used to control the visual sensor to adjust the image acquisition range according to the final adjustment instruction.

5. The device according to claim 4, It is characterized in that The temporary range is a rectangle.

6. The device according to claim 4 or 5, It is characterized in that The visual sensor includes a camera.

Images