System and method of detecting and locating objects surrounding a mobile vehicle
The LM pattern-based system addresses navigation challenges by simplifying object detection for mobile devices, improving reliability and speed, and reducing complexity, enabling effective detection of thin objects and minimizing blind spots.
Patent Information
- Application Number
- PCT/RU2025/050109
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2024-05-02
- Filing Date
- 2025-04-22
- Publication Date
- 2025-11-06
AI Technical Summary
Existing navigation systems for mobile devices, such as autonomous vehicles and robotic devices, face challenges including inaccurate object detection due to lack of surface features, poor performance in low light conditions, high power consumption, and complex image processing, as well as difficulties in detecting thin objects and large areas, leading to blind spots and increased system complexity.
A system and method using a localized moiré (LM) pattern of light generated by a wide-field, lensless projection device, which projects a spatially localized quasi-periodic moiré pattern to detect objects without triangulation or time-of-flight measurements, utilizing a projection device with periodic transmissive gratings and image recording devices to process 2D images for object detection.
This approach simplifies object detection, enhances reliability and speed, reduces blind spots, and allows for thin object detection, while minimizing system complexity and cost, enabling efficient navigation in various conditions.
Smart Images

Figure RU2025050109_06112025_PF_FP_ABST
Abstract
Description
[0001] TITLE OF THE INVENTION: System and method for detecting and determining the position of objects near a mobile device. FIELD OF THE TECHNOLOGY: The invention relates to the field of measurement technology and can be used to monitor the space surrounding a mobile device, in particular, to detect and determine the position of objects and obstacles near the mobile device, as well as to facilitate the navigation of the mobile device. PRIOR TECHNOLOGY: To perform the operational functions of autonomous and semi-autonomous mobile devices, such as vehicles and robotic devices, reliable localization and navigation are required. For navigation, in particular, it is necessary to detect and determine the position of objects and obstacles near and along the path of the mobile device, using sensors.Automatic object and obstacle detection enables fully autonomous vehicles, such as driverless cars, warehouse and agricultural equipment, and autonomous robotic devices, to navigate without collisions, getting stuck, damaging objects, or being damaged themselves. For semi-autonomous or controlled vehicles, such as remote-controlled mobile systems and vehicles with driver-assistance systems, automatic obstacle detection facilitates control by alerting the operator or driver to the presence of objects and obstacles or automatically adapting human commands. In particular, devices called parking sensors are designed to assist with parking vehicles and navigate busy roads and courtyards.To ensure safe movement and maneuvering in space, it is advisable to use contactless sensors and sensor systems capable of detecting obstacles and objects near and along the vehicle's path in real time. Such sensors and systems should preferably have high speed, low power consumption, and low cost. Passive contactless navigation systems for vehicle navigation are known, which include at least one image recording device, such as a television camera, designed to monitor the surrounding space, as well as a system for processing the resulting images. Stereovision is a widely used passive navigation method based on recording images of objects located in the vehicle's surrounding space (see, for example, [1-4]).In stereovision, images of the surrounding environment are recorded by at least two image sensors located in different locations on a moving vehicle, with their relative positions known. Distances to object points are determined using triangulation (parallax). The disadvantage of this method is that its implementation requires identifying the same areas or points of the object in images recorded by different image sensors. This is not always possible, so measurement results using passive stereovision systems can sometimes be erroneous. For example, errors can occur when the object lacks surface features such as corners, patterns, edges, etc., that can be easily identified in the images. Furthermore, stereovision systems perform poorly in low light conditions, such as at night in the absence of artificial lighting.Another disadvantage of passive stereovision systems is that they are generally not fast enough to localize objects and navigate in real time, and they require significant computing power. Numerous active methods for detecting and measuring the distance to objects and obstacles located near a moving vehicle are known. Some of these methods utilize ultrasonic sensors (sonars) (see, for example, [5-9]). In such an ultrasonic sensor, a transmitter generates and emits ultrasound, typically emitting short ultrasonic pulses. The receiver records the reflected or scattered echo signals. An ultrasonic sensor can detect objects at distances of up to several meters, and a single detector typically has a field of view of approximately 30 degrees. Using ultrasonic sensors for navigation has several disadvantages.In particular, their drawback is the difficulty in distinguishing true echo signals reflected from an object from noisy ultrasonic signals or multiple reflection signals. Ultrasonic sensors perform worse when the vehicle is moving over rough terrain, during rain or snowfall, when tall grass or bushes are in the sensor's line of sight, or when ice or other contaminants are present on the sensor surface. Furthermore, ultrasonic sensors typically need to be mounted high enough above the surface to avoid its field of view and the possibility of generating false echoes. Ultrasonic sensors have difficulty detecting and localizing potholes, curbs, thin objects such as cables, chains, ropes, pipes, poles, trees, as well as objects and objects that absorb ultrasonic waves (clothing, porous materials, piles of snow) or that reflect ultrasound primarily in the direction opposite the sensor.Even in cases where several ultrasonic sensors are installed on a mobile device, there are usually blind spots near it. Active methods for detecting and localizing obstacles and objects located near a mobile device are known, for the implementation of which electromagnetic radiation in the visible and near-visible ranges is used (see, for example, [1]). An active method for detecting and determining the position of objects and obstacles using infrared (IR) radiation is known (see, for example, [10 - 12]). To implement this method, at least one IR sensor is used, which contains an IR radiation source that forms a thin beam of IR radiation, and a detector sensitive to IR radiation. The IR radiation source is, for example, a light-emitting or laser diode with a lens or diaphragm at the output. The detector is usually a collimated IR photodiode.The detector records IR radiation reflected or scattered by objects located near a mobile device. The area illuminated by the IR source falls within the field of view of the IR detector. Using the IR sensor, objects intersecting the source and detector can be detected and their distances determined. Specifically, the distance from a mobile device to the road surface, floor, or wall can be monitored. An IR sensor typically detects objects at distances of approximately 1 meter or less. A significant drawback of such IR sensors is that they are difficult to monitor over large areas, as they can only detect objects located along a narrow IR beam. Rotating the IR sensor to expand the field of view increases measurement time, resulting in reduced system performance.Installing multiple IR sensors on a mobile device complicates the monitoring system, but does not completely solve the problem of monitoring large areas of space. Time-of-flight methods for detecting objects and obstacles, as well as measuring their coordinates, are widely used (see, for example, [1, 13-18]). These methods are based on measuring the propagation time of light from the light source to the object and back. Laser lidars are most often used for their implementation (see, for example, [1, 14-18]). Typically, the laser used in a lidar emits short-pulse or continuous modulated optical radiation. In lidar navigation systems, the laser and receiver are often mounted on a rotating platform to allow scanning of the surrounding space with a narrow laser beam. The quality of three-dimensional (3D) images obtained using a lidar is usually quite high. However, this device has a number of disadvantages.Lidars are power-hungry and expensive devices; they perform unreliably in rain or snow, and require extensive computing resources to generate a 3D image of the surrounding space. They are poorly suited for monitoring areas less than 3 meters away. They have difficulty detecting and localizing thin objects. Lidar radiation can be hazardous to the eyes and optical devices. Using lidars as parking sensors requires installing multiple devices on a vehicle, significantly complicating and increasing the cost of the navigation system. Furthermore, lidars are too bulky to be installed on small robotic devices.Systems and methods for detecting and determining the position of objects using the method of active stereovision (active optical triangulation) are known, which are based on the emission of structured or coded light (light patterns) by a projection device mounted on a mobile device and the recording of the image of the projection of such light structures onto the surface of objects located in the region of space illuminated by the projector (see, for example, [1, 19–33]). The active optical triangulation system for monitoring the space surrounding a vehicle, which is known from source
[0028] , is a prototype of the system proposed in the present invention.The prototype system comprises at least one projector capable of emitting infrared light into the space surrounding the vehicle, at least one IR camera capable of recording images of objects illuminated by infrared light, and a unit for processing images captured by one or more IR cameras. The infrared light projector includes an IR light source, as well as devices for forming and outputting an IR radiation beam. In the projection device, a light-emitting diode or a laser may be used as the IR radiation source. The device for forming the infrared light beam may, in particular, be a spatial light modulator, for example, a mirror based on microelectromechanical systems (MEMS)
[0028] . The IR radiation beam is output using a lens mounted at the output of the projector.Such a projector can emit either a smooth beam of light or structured light, such as one or multiple thin beams of light, a strip, or a pattern of stripes of light. The projector projects a smooth beam of light or structured light onto the scene and objects surrounding the vehicle. The image recording unit uses an IR camera to record the infrared light reflected from or scattered by objects, determining and displaying the two-dimensional (2D) luminance distribution of the reflected or scattered light. The projector and image recording unit are located on the vehicle at a known distance from each other. Typically, the optical axes of the projector and image recording unit are oriented at an angle to each other.The image processing unit, using the triangulation method, determines the distance to the object based on the intensity distribution of reflected or scattered IR radiation recorded by the image recording unit, as well as on the shape of the structured light beam emitted by the projector. The disadvantages of active triangulation navigation systems and, in particular, the prototype system are: • the use of a complex-to-manufacture spatial light modulator in the projector, as well as a high-quality lens, • the high cost of the optical projector, • the need to use powerful computers to calculate 3D images of the surrounding space • the need to calibrate the measuring system as a whole.When using active triangulation to detect and locate objects, the distance to the object's points and its shape are determined by the angles at which light structure elements are projected onto the object's surface and observed [1, 28, 34]. Active triangulation performs poorly when the projected light structure poorly matches the shape and surface structure of the object. For example, projected lines may be positioned at an inconvenient angle to the object's surface structure lines or the edges of the scene, which negatively impacts the accuracy and reliability of measurements. Some elements of projected optical structures may be difficult or impossible to detect in the image because they are obscured from the recording device, for example, by protrusions on the object.In addition, the disadvantages of the active triangulation method for monitoring the space around a mobile apparatus using structured or coded light are: • the complexity of image processing algorithms, • the need for additional uniform illumination to record 2D images of objects, • the dependence of the transverse size of the area illuminated by one projector on the distance to the projector, • the presence of blind spots near the projector, • the need to use several projection devices to monitor wide areas of space, • mutual interference that occurs when using several projectors. DISCLOSURE OF THE INVENTION Taking into account the disadvantages of the systems and methods for detecting and determining the position of objects near a mobile apparatus described in the Prior Art section, the invention is based on the objective of proposing a system and method that will eliminate at least some of these disadvantages.In particular, the objective of this invention is to provide an optical system and method for detecting objects and determining the position of objects near a mobile vehicle that do not utilize triangulation or time-of-flight measurement methods. More specifically, the objective of this invention is to provide a system and method that enable the detection and determination of the position of objects near a mobile vehicle using a localized moiré (LM) pattern of light generated by a wide-field, lensless projection device. The proposed system, like the prior art system, comprises at least one projection device (projector) emitting structured light. The projection device includes a light source and a device for generating and outputting the structured light.In a particular case, the proposed system, like the prototype system, comprises at least one device for recording images of a region of space illuminated by a projection device, as well as a computing device. In another particular case, the driver or operator controlling the vehicle observes and analyzes the image of the space illuminated by the projection device. Unlike the prototype system, in the proposed system, the projection device comprises at least one periodic transmissive grating, which is illuminated by one or a periodic set of light sources in the visible or near-visible range, which we will call the illuminator. The illuminator and the transmissive grating are spatially extended in at least one specified transverse direction, which is perpendicular to the normal to the grating surface. The output of such a projector does not require a lens.What's new is that the wide-field projection device used in the proposed system generates a light moiré pattern that is highly localized in the longitudinal direction, which is perpendicular to the surface of the transmission grating installed in the projection device. The proposed system also differs from the prototype system in that the longitudinal axis of the projection device and the optical axis of the image recording device can be parallel. In a specific case, the proposed system utilizes a projection device in which two flat one-dimensional (1D) periodic gratings are installed. These gratings are positioned one behind the other in non-coincident parallel planes on one side of a spatially extended illuminator and are illuminated by the illuminator. The second grating, which serves as the output grating, receives light transmitted through the first grating.The grating periods, the distance between their planes, the dimensions of the illuminator and gratings in the direction parallel to the plane of the output grating and perpendicular to the direction of its stripes, and the width of the angular spectrum of light emitted by the illuminator in a plane perpendicular to the direction of the grating stripes must be such that at least one LM structure is formed by the projection device. In another particular case, the system utilizes a projection device in which one flat periodic transmitting grating is installed, which serves as the output grating, and a grating illuminator, which is a periodic grating, or a linear array, or a matrix of light sources whose luminous surfaces are located in a plane parallel to the plane of the output grating, wherein at least one direction of the periodic change in the characteristics of the output grating and the grating illuminator coincides. The output grating is illuminated by the grating illuminator.The parameters of such a projection device and its elements must be such that they form at least one localized moiré pattern. The proposed method, like the method implemented using the prototype system, involves illuminating the spatial region surrounding the moving apparatus with structured light generated by at least one projection device. What's new is that the method utilizes structured light that represents at least one spatially localized, quasi-periodic moiré pattern. The locality of the moiré pattern means that the depth of field of this moiré pattern is at least four times smaller than the distance from the plane of the projection device's output grating to the plane parallel to it, where the contrast of this moiré pattern is greatest.In the visible and adjacent ranges of the electromagnetic spectrum, a localized moiré pattern of light can be formed, for example, using the projection device described in the invention. Unlike the method implemented using the prototype system, the proposed method does not employ triangulation to determine the distance to an object and its position. In the specific case of implementing the proposed method for detecting and determining the position of objects near a mobile device, images of the surrounding space and objects onto which structured light is projected are recorded using at least one recording device. A computing device then mathematically processes the resulting images, and based on the results of this processing, the position of the object is determined, and the fact that the mobile device is approaching the object at a known distance can also be recorded.The approach used for mathematical processing of 2D images of the surrounding area illuminated by a projection device differs from the approach used for processing data obtained using the prototype system. Specifically, the proposed method involves calculating the contrast of a quasi-periodic brightness change with a known spatial period, as a function of 2D image coordinates. The brightness changes are recorded by the image recording device at different relative positions of the mobile device and the object. When at least one region appears in the image within which the contrast of the quasi-periodic brightness change with this period exceeds a specified value, the approach of this region of the object's surface to the plane of the projection device's output grating at a known distance or closer is recorded.Next, using the computing device, the position of this region of the object relative to the mobile apparatus and its components is determined, taking into account that the plane of constant contrast, for example, the plane of greatest or equal to half the greatest contrast of the LM structure, is parallel to the plane of the projection device's output grating and is located at a known distance from it. Also, taking into account the results of the image recording device calibration, information about the mobile apparatus's shape, and the projection device's position on it. If the distance to the object is less than or equal to a predetermined value, the computing device or another control device may issue a command to stop or change the direction of the mobile apparatus, or a signal may be sent to the driver or operator controlling it.In another specific case, observation of a spatial region illuminated by a projection device forming at least one LM structure is performed by the driver or operator of a mobile vehicle. Upon visually registering at least one region on the surface of an object within which a quasi-periodic change in surface brightness with a known spatial period and visible contrast is observed, the driver detects the approach of the mobile vehicle to the object or part of it at a known distance. Based on this information, the driver or operator can decide to stop or change the direction of the mobile vehicle.The technical result is a simplification of the system for detecting and determining the position of objects near a mobile apparatus, including from the standpoint of manufacturing and setting up its devices, an increase in the speed and reliability of the system, a simplification of the method for detecting and determining the position of objects near a mobile apparatus and its navigation, the possibility of detecting and determining the position of thin objects, a decrease in the number and size of blind spots near a mobile apparatus, the possibility of combining the function of determining the position of objects near a mobile apparatus and illuminating an area of space near it. The essence of the invention is explained by the following figures. Fig. 1 (a, b) schematically shows a mobile apparatus, a system for detecting and determining the position of objects near a mobile apparatus, as well as an object located in front of it. In Fig. 1 (a), the object is located outside the localization area of the light moire structure formed by the projection device; in Fig.1 (b) the object is located in the region of localization of the moiré structure. Fig. 2 shows a diagram of a two-grating projection device. Fig. 3 shows a diagram of a projection device with a single transmission grating, in which the illuminator is a one-dimensional periodic array of light sources. Fig. 4 shows a diagram of a projection device with a single transmission grating, in which the illuminator is a periodic line of light sources. Fig. 5 shows a diagram of a projection device with a single transmission grating, in which the illuminator is a periodic rectangular matrix of light sources. Fig. 6 schematically shows the projection onto the y = 0 plane of the elements of the two-grating projection device, the image recording device, the object, and the localized moiré structure with the display of the positional and angular relationships between the elements of the devices, the light rays and the spatial regions of the LM structure. In Fig.7 schematically depicts a mobile apparatus, a system for detecting and determining the position of objects in front of and behind the mobile apparatus, which uses two projection devices and two image recording devices. Fig. 8 schematically depicts a top view of an object, a mobile apparatus, and a system for detecting and determining the position of objects near the mobile apparatus, which includes two projection devices whose optical axes are directed in the horizontal plane at an acute angle to each other, and two image recording devices. Fig. 9 schematically depicts a mobile apparatus and a system for detecting and determining the position of objects, which uses two projection devices that form LM structures with horizontally directed stripes, as well as objects located in front of the mobile apparatus.EMBODIMENT OF THE INVENTION The method for detecting and determining the position of objects near a mobile vehicle and the system for implementing it will be discussed with reference to the aforementioned figures, in which the same digital symbols denote the same elements of the system or the same elements of the spatial structure of the optical radiation. In these figures, the same letter symbols denote the same characteristics of the system and its elements or the same characteristics of the optical radiation. The system and method proposed in the invention can be used to assist in the navigation of mobile vehicles of various types and designs. The method, in a particular case of its implementation, for example, in the case of its use to assist in the navigation of an unmanned autonomous mobile vehicle, can be implemented using the system, a schematic representation of which is shown in Fig. 1 (a, b).The system for detecting and determining the position of objects, which is installed on the mobile apparatus 1, includes at least one projection device 2, at least one image recording device 3 with a lens 4 and a computing device 5. For the sake of definition, we will assume that the mobile apparatus 1 moves on a solid surface. It is biaxial and is driven by wheels. The rotation of the mobile apparatus is achieved by turning its front wheels 6, and the direction of the rear wheels 7 relative to the mobile apparatus is fixed and determines the direction of its rectilinear motion. In Fig. 1 (a, b), an object 8 is also shown, which is illuminated by the light emitted by the projection device 2. In Fig. 1 (a), the object 8 is located outside the localization region of the light LM structure formed by the projection device 2. In Fig. 1 (b), the object 8 is located in the localization region of the moire structure.The projection of the LM structure onto the surface of object 8 is designated by the number 9. A two-grating projection device that forms a localized moire structure is known from patent RU2807409
[0035] . In this patent, such a projection device is used to implement a method for contactless ranging and profilometry
[0035] . The two-grating projection device described in
[0035] is one of the special cases of a projection device that is proposed to be used in the present invention. The diagram of the two-grating projection device 2 is shown in Fig. 2. The projection device 2 includes a light source 10, which will also be called an illuminator, and two parallel flat one-dimensional periodic transmission gratings 11 and 12. The stripes of these gratings have the same direction. The first grating 11, closest to the illuminator 10, and the second, output grating 12 are located on the same side of the illuminator 10.The illuminator 10 and the gratings 11 and 12 are spatially extended in the direction perpendicular to the normal to the surfaces of the gratings and the direction of their stripes. It is preferable that the gratings 11 and 12 have a rectangular shape. The illuminator 10 and the gratings 11 and 12 can be directly fixed in the housing of the projector 2 or inserted into frames (not shown in the figures), which are fixed in the housing of the projector 2. In a particular case, the housing of the projection device 2 or the inner surface of this housing can be made of a material that absorbs light well. The luminous surface of the illuminator 10 faces the gratings 11 and 12. The illuminator 10 emits incoherent light in at least one range belonging to the visible region and the spectral regions adjacent to it. In a particular case, the extended light source 10, installed in the projection device 2, is a spatially homogeneous source of diffuse light.Such a diffuse light source may be, for example, a fluorescent lamp, a fluorescent or LED screen, the surface of an organic light-emitting diode, the surface of a random phase transparency, such as frosted glass, a sheet of tracing paper, or tissue paper, illuminated from the side opposite the gratings. The light source 10, which is installed in the projection device 2, may also be a light-emitting screen of a display, monitor, telephone, or television. In a particular case, the projection device 2 may employ an illuminator 10 whose temporal characteristics, such as the duration and duty cycle of the light pulses, can be controllably varied. Fig. 1 (a, b) and Fig. 2 shows a right orthogonal coordinate system with axes x, y and z, which is connected with projection device 2. The coordinate axis z is directed perpendicular to the planes of gratings 11 and 12, in the direction from the light source 10 to gratings 11 and 12.If the second grating 12 has a rectangular shape, then the z-axis passes through its center, and is hereinafter called the optical axis of the projection device 2. The y-axis is directed along the direction of the stripes of gratings 11 and 12 (see Fig. 2). The transmittances of gratings 11 and 12 depend periodically on the x-coordinate. The spatial periods of the first and second gratings are equal to p 1 and p 2, respectively. The direction of the z-axis will be called longitudinal hereinafter, and the directions perpendicular to the z-axis will be called transverse. We will refer to the dimensions of the projection device 2, the working apertures of the light source 10, gratings 11 and 12, etc. in the direction of the x-axis as their width, in the direction of the y-axis as their height, and in the direction of the z-axis as their length or thickness. The plane z ^ 0 coincides with the plane of the first grating 11. The second, output, grating 12 is located in the plane z ^ L 1 (see Fig. 2), i.e. the planes of the first and second gratings are removed from each other by a distance. We assume that the working surface of the illuminator 10 is parallel to the planes of the gratings 11 and 12, and that it and the gratings 11 and 12 have the shape of rectangles whose sides are parallel to the x and y axes (see Fig. 2). The width of the working aperture of the illuminator 10 is equal to H x , its height is equal to H y . The width of the working aperture of the grating 11 and the grating 12 is equal to D 1x and D 2x , respectively, and their height is equal to D 1y and D 2y , respectively. The centers of the working surface of the illuminator 10, as well as the first grating 11 and the second grating 12 are located on the optical axis of the projection device, i.e. the centers of these elements of the projection device 2 have transverse coordinates x ^ 0, y ^ 0. The distance in the longitudinal direction between the working surface of the illuminator 10 and the plane of the first grating 11 will be designated as L 0. It can be equal to zero. The distance in the longitudinal direction from the second grating 12 at z ^ L 1 will be called the distance from the projection device 2. In Fig.1 (a, b) also shows a global fixed coordinate system with axes x^, y^ and z^. In a particular case, a device for determining the coordinates and orientation of the mobile apparatus 1 in a global coordinate system, for example, in the GPS or GLONASS system, can be additionally used. In a particular case, a device can be used in the projection device 2 that controllably changes the distance between the gratings 11 and 12 in the longitudinal direction or brings the gratings 11 and 12 closer together or further apart in this direction at a given speed. Additionally, there can be a device for the translational movement of at least one of the gratings 11 and 12 in its plane in the direction of the x coordinate, which leads to a change in the phase of the quasi-periodic moire structure formed by the projection device 2 (see
[0035] ). Devices that can move the gratings 11 and 12 in different particular cases of the projector implementation are not shown in the figures.The amplitude transmittances of the gratings installed in the projection device 2 may generally be complex. In a particular case, at least one grating having a real positive amplitude transmittance, called an amplitude grating, may be installed in the projection device 2. In a particular case, at least one grating installed in the projection device 2 is a 1D amplitude grating with a sinusoidal dependence of the magnitude of change in the amplitude transmittance on the x-coordinate [35 - 37]. In another particular case, at least one grating may be a 1D amplitude grating that has a binary dependence of the transmittance on the transverse coordinate, for example, it is periodically alternating transparent and opaque parallel rectangular stripes along the x-coordinate, which have sharp boundaries (see Fig. 2) [35, 38 - 45].Such gratings are also called periodic binary rasters. The filling factor of the binary raster is equal to alpl, where pl is the grating period, al is the width of its transparent stripes, here the index l, equal to 1 or 2, denotes the ordinal number of the grating in the projection device 2. An amplitude binary raster with al pl ^ 0.5 is also called a Ronchi grating. The advantage of using periodic binary rasters in a projection device is the ease of manufacturing these gratings and their commercial availability. In yet another particular case, at least one phase grating can be used in the projection device 2, i.e., a grating with a periodic dependence of the phase of the amplitude transmittance on the transverse coordinate and a constant modulus of this coefficient [42, 46]. As yet another alternative, at least one grating combining an amplitude and a phase grating can be used in the projection device [42, 47, 48].Amplitude, phase, and amplitude-phase gratings are produced by known photographic, holographic, lithographic, printing, and other methods. Gratings of some of the types mentioned are commercially available [43–46]. In a particular case, at least one grating whose characteristics can be changed can be used in the projection device 2. For example, at least one of the gratings installed in the optical projection device 2 can be an amplitude binary raster whose fill factor and / or period can be changed. In a particular case, a spatial light modulator can be used to form at least one of the gratings installed in the projection device 2. The distance between the working surface of the illuminator 10 and the plane of the first grating 11 can be equal to zero, and their role can be performed by one element of the projection device 2, which will be called a grating illuminator.The grating illuminator emits incoherent light in at least one spectral range belonging to the infrared, visible and ultraviolet regions. Fig. 3, Fig. 4 and Fig. 5 show the diagrams of the projection device 2 in particular cases of its implementation, when one transmitting one-dimensional periodic grating 12 is installed in the projection device 2, which is the output one, and a grating illuminator 13 is used in it, in which the luminous surfaces of the individual light sources are periodically located in a plane parallel to the plane of the output grating 12. This plane is removed from the plane of the output grating 12 by a distance L 1 ^ 0. Fig. 3 - Fig. 5 show a right-hand orthogonal coordinate system with axes x, y and z, which is associated with the projection device 2. The longitudinal coordinate axis z is directed from the grating illuminator 13 to the grating 12 perpendicular to its plane. The plane z ^ 0 coincides with the luminous surface of the grating illuminator 13.The output grating 12, which is installed in the projector 2, can be an amplitude, phase or amplitude-phase grating. The spatial period of the grating illuminator 13 p 1 is greater than the width of the working surface of the individual light source, which is designated as a 0. The width of the working aperture of the illuminator 13 is equal to H x , its height is equal to H y (see Fig. 3, Fig. 4 and Fig. 5). Fig. 3 shows a diagram of the projector 2, in which the illuminator 13 is a one-dimensional periodic grating of relatively narrow light sources 14 (a 0, p 1 ^^ H y ) [49 - 52], in which the stripes of the luminous surfaces are directed parallel to the stripes of the output grating 12.As such an illuminator, one can use, for example, a light-emitting 1D periodic grating on the flat surface of a semiconductor light-emitting diode, described in patents [50 - 52], as well as a 1D periodic light-emitting structure displayed on the surface, for example, of a liquid crystal, LED, plasma or fluorescent screen of a monitor, display, telephone, television, etc. In Fig. 4, a diagram of a projection device 2 is shown, in which an illuminator 13 is used, which is a periodic line of light sources 15, the centers of the luminous surfaces of which are located on a line parallel to the plane of the output grating 12 and perpendicular to the direction of its stripes. In Fig.5 shows a diagram of a projection device 2, in which an illuminator 13 is used, which is a rectangular matrix of light sources 16, the luminous surfaces of which are arranged periodically in a plane parallel to the plane of the output grating 12, and the columns of the matrix of light sources are parallel to the direction of the stripes of the grating 12. The rectangular matrix of light sources 16 can also be oriented so that its rows or diagonals of the rectangles, at the vertices of which the four light sources (matrix elements) 16 closest to each other are located, are parallel to the direction of the stripes of the output grating 12
[0053] . In the illuminator 13, which is a line or matrix of light sources, individual light sources 14 - 16 can serve, for example, as light-emitting diodes.In particular cases, a grating illuminator 13 may be used in the projection device 2, the spatial characteristics of which may be controllably changed, for example, the period, duty cycle, and / or temporal characteristics, for example, the duration and / or duty cycle of light pulses. The spatial and temporal characteristics of the grating illuminator may be changed, for example, by the methods described in [52, 53]. In particular cases, a display screen, monitor, telephone, television, etc. may be used in the projection device 2 as a controllable or uncontrollable grating illuminator 13. In particular cases, a device may be used in the projection device 2 that controllably changes the distance in the longitudinal direction between the output grating 12 and the grating illuminator 13 or brings them closer together or further apart in this direction at a given speed.Additionally, there may be a device for the translational movement of the output grating 12 or the grating illuminator 13 in the direction of the x coordinate. The devices that, in different particular cases of the projector implementation, can move the output grating 12 or the grating illuminator 13, are not shown in the figures. In the particular case of the implementation of the system for detecting and determining the position of objects near the moving apparatus, observation of the region of space, which is illuminated by the projection device 2, is carried out by the driver of the vehicle. The image of this region is recorded by him "with the naked eye". In the preferred particular case of the implementation of the system for detecting and determining the position of objects near the moving apparatus, the recording of the image of objects located in the region of space, which is illuminated by the projection device 2, is carried out by at least one image recording device 3 (see Fig. 1 (a, b) and Figs. 6 - 9).The image recording device 3 may be, for example, a television camera or a photo camera. The lens 4 of the image recording device 3 projects the light incident on it onto the image sensor 17 (see Fig. 6). The image sensor 17 in the image recording device 3 is, for example, a CCD (charge-coupled device) or CMOS (complementary metal-oxide-semiconductor) matrix. An optical filter may be installed in front of the lens 4 of the image recording device 3, which, for example, transmits light only in the spectral range in which the light is emitted by the projection device 2. The optical axis of the image recording device 3 may be parallel to the longitudinal axis of the projection device 2. Before using the image recording device 3 for implementing the proposed method, it is preferable to calibrate it using known methods, for example, using a checkerboard-shaped template [54, 55].In a particular case, the system additionally uses a display (not shown in the figures), on the screen of which the images recorded by the image recording device 3 are displayed. In a particular case, the projection device 2 and / or at least one image recording device 3 may be rigidly fixed on the movable apparatus 1. In another particular case, they may rotate relative to the movable apparatus 1 and / or move along it translationally. The devices by means of which the position and orientation of the projection device 2 and / or the image recording device 3 are changed, as well as the position of individual elements of the projection device 2 are changed, are not shown in the figures. They are generally known and commercially available. The computing device 5 is intended for the mathematical processing of the images obtained by the image recording device 3.The computing device 5 is, for example, a computer, a mobile computing device, or a programmable integrated logic chip (FPGA). In addition to storing and processing information received from the image recording device 3, the computing device 5 can process and store data received from other devices of the system and other devices of the mobile apparatus, as well as control them and their elements. In particular, it can process and store data on the displacement values and rotation angles of the projection device 2, the image recording device 3, as well as on the displacement values of the gratings 11 and 12, and the illuminator 13, which are received from the devices recording these values. The computing device 5 is connected to other devices of the object detection and position determination system and other devices of the mobile apparatus via wires and / or wirelessly.It can also be connected to other computing devices, in particular to external computing devices, for example via mobile Internet, Wi-Fi or Bluetooth. In a particular case, computing device 5 can be connected to and can control a display (not shown in the figures), which is intended for displaying images recorded by image recording device 3. In a particular case, images from image recording device 3 can be transmitted to the display after processing by computing device 5. Computing device 5 can be connected to an audio device, which can be installed on mobile device 1 (not shown in the figures) and is intended to warn the driver with an audio signal about the presence of an object or obstacle near mobile device 1. In a particular case, when two parallel one-dimensional amplitude gratings 11 and 12 are installed in projection device 2 (see Fig. 2 and Fig.6), the lines of which are directed parallel, the projection device 2 operates as follows. Light emitted by illuminator 10 passes first through first grating 11, then through second grating 12. As light passes through the amplitude grating, its intensity is spatially modulated, so that periodic bands of light and shadow are observed behind each of gratings 11 and 12. When the grating is illuminated by incoherent light from an extended source or by diffuse light, the contrast of the bands of light and shadow decreases with increasing distance from the transmission grating in the longitudinal direction. At a sufficiently large distance from the grating, the light intensity becomes almost constant in space.However, when light passes through a pair of parallel gratings 11 and 12, there are planes parallel to the planes of the gratings and located at known distances from them in the longitudinal direction, in the vicinity of which the projection device 2 can form light moiré structures, which are periodic in space regions of light and shadow [35, 38-40, 56-63]. When these light moiré structures are projected onto a plane parallel to the planes of the gratings, periodic light and dark bands are observed, which are directed along the y-axis [35, 40]. Such a LM structure is also called a pseudo-image of the grating [39, 57,58, 60-62] and a shadow echo [35, 63, 64]. The moiré structures formed by a two-grating projection device with 1D gratings are designated by two indices (N, M), where N and M are natural numbers. The distance from the plane of the output grating 12 of the projection device 2 to the plane where the contrast of the moire structure (N, M) is greatest is calculated using formula (1). where RNM ^ Mp 1 Np 2 , p 1 and p 2 are the spatial periods of the first and second lattices, respectively [35, 38, 58 - 60]. For a moiré structure to arise (N, M), the value must be greater than 1. The coordinate of the plane where the contrast of the moiré structure (N, M) has the greatest value is equal to z ^ z NM , where This plane will be called the plane of greatest contrast of the localized moiré structure (N, M). The dependence of the light intensity on the x coordinate near the plane of greatest contrast of the moiré structure (N, M) z^ z NM is described by a periodic function usually close to a sinusoid, the spatial period P NM of which is calculated by the formula [35, 38, 58 - 60]. (3) The contrast (visibility) of the moire structure (N, M) is calculated using the formula where I max and I min are the maximum and minimum values of light intensity in the z ^ const plane [35, 60, 62]. The distance in the x-axis direction between the points at which the quantities are determined is equal to P NM , where n is a natural number, including zero. The contrast of the moiré structure (N, M) depends on the longitudinal coordinate z, and this dependence is described by the formula
[0035] where CNM ( z NM ) is the contrast value of the moire structure (N, M) in the plane z^ z NM . The value of the function at z^ z NM is maximum and equal to 1. Spatial distribution of light intensity in the y^ 0 plane near the z^ line условноshown in Fig. 6 and designated by the number 18. The full width at half maximum (FWHM) of the function WNM ^^ z NM ^ is equal to the depth of field ^L NM of the moire structure (N, M), where ^ z ^ z NM . The depth of field ^L NM is equal to the doubled value of the length at which, when removed in the longitudinal direction from the plane z^ z NM, the contrast of the moire structure (N, M) decreases by half compared to the value of its contrast in the plane z^ z NM . In a certain range of parameters of the projection device 2 and its elements, it forms light moire structures that are relatively strongly localized in the longitudinal direction
[0035] . The locality of the moire structure (N, M) means that its depth of field ^L NM is significantly less than the distance from the projection device 2 to the plane on which the contrast of this moire structure has the greatest value, i.e. the relative depth of field of the moire structure (N, M) ^LNM L NM ^^ 1 , for example, less than 0.25.The width of the LM structure (N, M), formed by a projection device with two thin 1D amplitude gratings 11 and 12, in the plane z^ z NM is equal to
[0035] (see Fig. 6). where ^ xm is the half-width in the y ^ 0 plane of the angular spectrum of light emitted by small areas of the working surface of the illuminator 10 with a uniform flat luminous surface. The localized moiré structure (N, M) has the greatest contrast and the smallest depth of field near the x ^ 0 plane. In the vicinity of this plane there is a region of width (see
[0035] ), within which the contrast and depth of field of the LM structure (N, M) do not change or change little, for example, less than 1.2 times, when x changes. The width of this central region of the LM structure (N, M) D^ N M can be estimated using the formula
[0035] If the projector housing or the frames limiting the first and second gratings do not vignette the illuminator 10 in the direction of the x coordinate when it is observed from the central region of the LM structure (N, M), which occurs, in particular, if the width of the working aperture of gratings 11 and 12 is greater than or equal to the width of the working aperture ^ ^ ^ depth of field of the LM structure (N, M) in the region
[0035] where ^ Ex is the smallest of the quantities ^ xm and ^ sx , where ^ sx is the angular size of half the working aperture of the illuminator in the plane y ^ 0 when observed from a point with coordinates x ^ 0 , y ^ 0 and z ^ z NM , equal to ^ H x ^ ^ sx ^ arctg ^^ . (9) ^ 2 ^ ^ L0 ^ L 1 ^ LNM ^ ^ ^The same estimation of the depth of field of the moiré structure in the case of ^ xm ^ ^ sx was made in [59, 61, 65]. As follows from formulas (6) and (7), by increasing the width of the illuminator, as well as the width of the gratings, it is possible to increase the width of the LM structure (N, M). The possibility of increasing the width of the LM structures is limited by the permissible width of the projection device. Thus, the system for detecting and determining the position of objects proposed in the invention is wide-field. Formulas (6) - (8) allow an approximate estimation of the width of the moiré structure (N, M), the width of its central region and the depth of field. To improve the accuracy and reliability of the proposed method, it is preferable to preliminarily measure the dependence of the contrast of the moiré structure (N, M) on the longitudinal coordinate z and the transverse coordinate x, and, based on the data of these experiments, find ^LNM ^ x ^ and determine the measurement error of the distance to the objects and their position.The error in determining the longitudinal distance from the projection device 2 to the object 8 or some part of it is proportional to the depth of field of the LM structure and usually does not exceed the value ^LNM ^ x ^ . To implement the proposed method, it is preferable to use a LM structure with a relative depth of field less than 10%. A localized moiré structure can be formed in the geometric optics (GO) zone and in the Fresnel diffraction zone of the projection device 2. The influence of diffraction on gratings 11 and 12 on the characteristics of the LM structure (N, M) can be ignored if the first and second gratings are located close to each other, so that the distance between their planes satisfies the condition (see [35, 63, 64]). where the length is calculated using the formula the value of ^ m is equal to ^ m ^ ^ 0 ^^ ^ , where ^ 0 is the central wavelength of the light emitted by the illuminator, ^^ is the half-width of the light spectrum by wavelength at the e^ 2 level. In the case when the parameters of the two-grating projector 2 and the parameters of its elements are such that the influence of light diffraction on amplitude gratings 11 and 12 on the characteristics of the LM structures is negligible, the geometric optics approximation is applicable to describe the characteristics of the moiré structures formed by such a projector. In the geometric optics approximation, the characteristics of the moiré structures do not depend on the wavelength of light, therefore, a source of both monochromatic and non-monochromatic light, in particular, a source of broadband light, for example, white light, can be used as an illuminator in the projection device. In the GO zone, the contrast of the LM structure does not depend on the distance between the gratings.A grating device that forms moiré structures (pseudo-images of the grating) in the Fresnel diffraction zone is called a Talbot-Lau interferometer in a number of publications [47, 66, 67]. To achieve high contrast of the LM structure (1,1) in the Fresnel diffraction zone, the distance between gratings 11 and 12 must be within limited intervals, the number of which is = 1, 2, 3, … increases sequentially with the growth of the distance between the gratings L 1 , while the half-width along the wavelength of the spectrum of light emitted by the illuminator 10 must be less than 0.4^ ^ 1 0 ^ 1[35, 60, 61]. For example, at p 1 ^ p 2, for a higher contrast of the LM structure (1,1), it is preferable that the distance between the gratings L 1 be in the range from ^^1 ^ 0.2 ^ LT ^ ^ 0 ^ to ^^1 ^ 0.2 ^ LT ^ ^ 0 ^ . Formulas (1) - (11) can also be used to evaluate the characteristics of the LM structures created by a projection device with one grating 12 and a grating illuminator 13. In this case, ^ xm is the half-width in the y ^ const plane of the angular spectrum of light emitted by an individual light source of the grating illuminator. Several LM structures can be simultaneously formed by one projection device. It is preferable to use LM structures with a sufficiently high contrast for implementing the proposed method, for example, the highest contrast value of such an LM structure should exceed 0.2.In the particular case where the first grating 11 and the second grating 12 in the projection device 2 are Ronchi gratings (al pl ^ 0.5), relatively contrasting moiré structures (1,1), (1,3) and (3,1) can be formed in the geometric optics region. The condition for the occurrence of each of them is the inequality Mp 1 ^ Np 2 , here N and M are equal to 1 or 3. The greatest contrast of the moiré structure (1,1) at p 1 ^ p 2 in the GO zone is approximately equal to C11. ^ 0.9. The highest value of contrast of moire structures (1,3) and (3,1) is equal to approximately 0.3. The remaining moire structures (N, M), formed by the projection device with two Ronchi gratings, have a contrast of less than 0.2 in the GO zone. Let us give a numerical example in which we determine the characteristics of moire structures formed by projection device 2 with two thin amplitude gratings 11 and 12 (see Fig. 2), for example, with Ronchi gratings. We assume that the periods of gratings 11 and 12 are equal to p 1 = 1.2 mm and p 2 = 1 mm, respectively, and the modulation depth of the transmission function of these gratings is equal to C 1 ^ C 2 = 1. The distance between the first grating 11 and the second grating 12 is equal to L 1 = 40 cm. Illuminator 10 is a spatially homogeneous light source. It is located close to the first grating 11, i.e. L 0 ^ 0. The illuminator 10 and the first grating 11 have equal width Hx ^ D 1 x = 40 cm. The second grating 12 has a width D 2x = 50 cm. The illuminator 10 emits diffuse light in the range ^ ^ 0.5 ^ 0.1 μm.The half-width in the y ^ 0 plane of the angular spectrum of light emitted by small incoherent elements of the illuminator surface, ^ xm, is equal to 10°. Projection device 2 with such parameters forms two moiré structures with a contrast greater than 0.2: these are moiré structures (1,1) and (1,3). Distances. between gratings 11 and 12 is less than the distance 0.2L T ^ ^ m ^ = 48 cm, therefore the influence of light diffraction on the characteristics of the moiré structure (1,1) can be neglected. The spatial period of the moiré structure (1,1) is 6 mm. This moiré structure will have the greatest contrast C11 ^ 0.9 in the plane located from the second grating 12 at a distance ^ 200 см. The width of the LM structure (1,1) is equal to D^ 11 ^125 cm, the width of its central region is ^44.6 cm. The depth of field of the moire structure (1,1) in its central region is ^4.3 cm. The relative depth of field of the moire structure (1,1) ^L 11 L 11 is approximately equal to 2%. The shape of the region on the surface of object 8, within which the projection 9 of the LM structure has a relatively high contrast, depends on the size and shape of the surface of object 8, as well as the depth of field of the moire structure (N, M) (see Fig. 1(b) and Fig. 6). The plane z ^ const, on which the contrast of the moire structure (N, M) is constant, intersects the surface of object 8 along a flat line called the isoline of constant contrast of the LM structure. The plane of constant contrast intersects flat surfaces along segments of straight lines
[0035] . In particular, the plane z^ z NM, where the contrast of the moire structure (N, M) is the greatest, intersects with the surface of object 8 along a flat line, which we will call the isoline of the greatest contrast of the LM structure. Planes z^ and z^ zNM ^ 0.5 ^^ L NM , at which the contrast of the LM structure is equal to we will call the near and far (to the projection device 2) planes of the half-contrast of the LM structure (N, M), respectively. These planes intersect with the surface of the object 8 along flat lines, which we will call the near and far isolines of the half-contrast of the LM structure. Let us consider the functioning of the system and the implementation of the method for detecting and determining the position of objects near the moving apparatus in the particular case when • the projector 2 on the moving apparatus 1 is installed so that the stripes of the output grating 12 are directed vertically (see Fig.1 (a, b)); • the position of the projection device 2 and the image recording device 3 on the movable apparatus 1 does not change; • the direction of the optical axes of the projection device 2 (the z axis) and the image recording device 3 coincide with the direction of rectilinear movement of the movable apparatus 1; • the parameters of the projection device 2 and its elements are such that it forms a contrast (with a contrast greater than 0.2) LM structure (N, M), the characteristics of which are known and do not change.We also assume that the object 8 located in the path of movement of the moving apparatus 1 is, firstly, sufficiently wide, such that the width of the projection onto the plane z^const of the part of its surface facing the output grating 12 of the projector 2 is at least twice as large as the period P NM of the LM structure used, and, secondly, its surface facing the moving apparatus 1 is located at a not very large angle to the plane of the output grating, such that the tangent of the angle between the surface of the object and the plane z^z NM in the areas of their intersection is less than. It is preferable that the plane with a given value of LM contrast P NM of the (N, M) structure, for example, the far plane of its half-contrast, is located at a distance from the moving apparatus 1 that is at least greater than the minimum necessary to avoid collisions with objects that may be in the path of the moving apparatus 1 when it moves at a given, for example, maximum, speed.When the above conditions are met, the method for detecting and determining the position of an object 8 near a moving apparatus 1 during its movement is carried out as follows. The area in front of the moving apparatus 1 is illuminated with light emitted by a projection device 2. The image recording device 3 is focused approximately on the plane of greatest contrast of the LM structure or another selected plane parallel to it in the interval from. .It is preferable that the depth of field of the lens 4 of the image recording device 3 is at least twice as large as the depth of field of the moire structure (N, M). It is also preferable that the entire LM structure (N, M) falls within the field of view of the image recording device 3. Otherwise, it is preferable to use several image recording devices. During the translational movement of the movable apparatus 1 and / or its rotation, the relative position of the projection device 2 and the object 8 changes. At each position of the movable apparatus 1, the image of the region of space illuminated by the projection device 2 is recorded by the image recording device 3. It is preferable that the time of recording one image by the image recording device 3 (one frame or a working series of frames) is at least several times less than the value ^L NM V , where V is the speed of movement of the apparatus 1 at a given moment in time.Using the computing device 5, the brightness distributions of the images recorded by the image recording device 3 at different positions of the mobile apparatus 1 are determined in real time. Then, the dependences on the 2D coordinates in the images of the contrast value of the quasi-periodic change in the brightness of these images with a spatial period and direction of the stripes corresponding to the LM structure (N, M) are calculated in real time. Known mathematical methods of image processing are used to determine the 2D distribution of the contrast value. In particular, Fourier or wavelet analysis of images, as well as the phase step method [34, 35, 67, 68] can be used. If the object 8 is removed from the plane of greatest contrast of the LM structure (N, M) by more than 2^L NM , then the quasi-periodic spatial modulation with a period P NM will be practically absent in the images (see Fig. 1(a)).When at least one region appears in the image within which vertically directed light and dark stripes are observed (see Fig. 1(b)), and a quasi-periodic change in brightness in the direction of the x-axis occurs with a period corresponding to the period of the LM structure P NM , and a contrast exceeding a given value, the fact of approaching the plane of the output grating 12 of the projector 2 to this region of the surface of the object 8 at a known distance is recorded. For example, if the contrast of the projection of the LM structure (N, M) onto the surface of the object 8 exceeds half of its greatest contrast, i.e. more than 0.5^CNM ( z NM ), then the distance from the plane of the output grating 12 of the projector 2 to this region of the surface of the object 8 does not exceed z^ zNM ^ 0.5 ^^ L NM .In a particular case of implementing the method, when registering such a contrast projection of the LM structure (N, M) onto the object 8, the computing device 5 or another computing device connected to it can give a command to stop or change the direction of movement of the movable apparatus 1. This particular case can be realized, for example, when the distance from the movable apparatus 1 to the far plane of half-contrast of the LM structure, equal to z^ zNM ^ 0.5 ^^ L NM , only slightly exceeds the minimum distance necessary to avoid a collision of the movable apparatus with the object 8. In a more preferred particular case of implementing the method, the coordinates of the region of the surface of the object 8 are determined, within which a contrast quasi-periodic change in brightness with a spatial period and direction of stripes corresponding to the LM structure (N, M) is observed in the image.In this case, it is taken into account that the planes on which the contrast of the LM structure is constant are parallel to the plane of the output grating 12 of the projection device 2. The longitudinal coordinate of the points of the surface of object 8, located, for example, on the isoline of half the contrast of the LM structure (N, M) closest to the moving apparatus 1, is equal to z ^ z NM ^ 0.5 ^ ^ L NM , on the isoline of greatest contrast it is equal to z ^ z NM , and on the far isoline of half the contrast it is equal to z ^ z NM ^ 0.5 ^ ^ L NM . The transverse coordinates of the points located, for example, on the isoline of greatest contrast of the LM structure, are determined using the value of their longitudinal coordinate. а такжеthe coordinates of these points on the 2D image of the surface of object 8, recorded by image recording device 3, also use the results of calibration of image recording device 3 [54, 55]. In the particular case when the coordinates of lens 4 of image recording device 3 in the coordinate system associated with projection device 2 are equal to x ^ 0, y ^ Y, z ^ ^ LC (see Fig. 1 (a, b) and Fig. 6), the coordinates x and y of the points on the surface of object 8 lying on the isoline of greatest contrast are calculated using the formulas where F is the focal length of the objective 4 of the image recording device 3, x C and y C are the coordinates of the images of these points on the surface of the image sensor 17, L CNM is the distance from the objective 4 of the image recording device 3 to the plane of greatest contrast of the LM structure (N, M), equal to LCNM ^ LC ^ L 1 ^ L NM (see Fig. 6). Using the found values of the coordinates of the region of the surface of object 8 onto which the LM structure (N, M) is projected, and also taking into account the data on the shape of the moving apparatus 1 and the position of the projection device 2 on it, the position of this region of the object 8 relative to the moving apparatus 1 and its parts is determined. The method described above can detect the presence of objects or their parts in front of the moving apparatus 1 in an area of approximately . Their position can be determined with the greatest accuracy in a region of width D ^ N M. If the object 8 is in the path of the moving apparatus 1 and the distance to it in the longitudinal or transverse direction is less than or equal to the specified one, then the computing device 5 or another device associated with it can give a command to stop or change the direction of movement of the moving apparatus 1. In a particular case, the computing device 5 can give a command to turn on the sound device, which will give a sound signal warning of the presence of the object 8 in dangerous proximity to the moving apparatus 1. In another particular case, the computing device 5 can give a command according to which on the screen of the monitor displaying the images obtained by the image recording device 3, the area with the contrast projection of the LM structure will be highlighted in the image, for example, in color or enclosed in a frame. When the object 8 or part of it is in the area of localization of the light moire structure (see Fig. 1(b) and Fig.6), and this part of the surface of the object 8 is not a plane perpendicular to the optical axis of the projector 2, then when the mobile apparatus 1 moves, the projection of the LM structure 9 will move along the surface of the object 8, if it is wide enough. In particular, when the mobile apparatus 1 approaches the extended object 8, the line of constant contrast of the projection of the LM structure will move along the surface of the object 8 from the area of the surface closer to the mobile apparatus 1 to more distant ones. When the mobile apparatus 1 moves, for example, during its rectilinear forward movement, it is possible to determine and, if necessary, store in the memory of the computing device 5 the coordinates in the global coordinate system of the points of the frontal part of the surface of the object 8 or a certain area thereof, in particular, it is possible to determine the coordinates of the part of the object 8 located closest to the mobile apparatus 1. This information can be used to ensure the safe navigation of the mobile apparatus 1.For example, based on this data, computing device 5 can issue a command to stop mobile device 1 from approaching object 8 before they reach a dangerous distance. Background illumination of objects, such as sunlight or headlights, negatively impacts the reliability and accuracy of the proposed method for detecting and determining the position of objects near a mobile device. The influence of background illumination on the results of measuring the contrast of quasi-periodic changes in the object's image brightness can be reduced by subtracting the image brightness distribution recorded with the projector on, i.e., with the projection device off, from the image brightness distribution recorded with the projector on. This measurement method can be implemented, for example, by implementing a pulsed or pulse-periodic mode of projector operation.The influence of background illumination can also be reduced by using an illuminator that emits light in a spectral range in which the intensity of background light is low (see
[0069] ) and by installing an optical filter in front of the lens 4 of the image recording device 3 that transmits light only in this spectral range. The projection device 2, in addition to forming light LM structures, can also be used for uniform illumination of a part of the space near the mobile apparatus 1 that is outside the area of localization of moire structures. With such illumination, without the use of additional light sources, it is possible to record two-dimensional images of objects or their parts that are preferably outside the area of localization of moire structures (see Fig. 1(a)). The recording of these images can be performed by the image recording device 3 or other image recording devices installed on the mobile apparatus 1.To increase the monitored spatial area in a particular case, a system can be implemented in which projection device 2 rotates in a circle or pivots within a given sector, for example, around a vertical axis. Image recording device 3 can also rotate around this axis or a parallel axis, in particular, rotating synchronously with projection device 2, so that the LM structure is within the field of view of image recording device 3. A servo drive, for example, can be used to rotate projection device 2 and / or image recording device 3. However, the design of a system with a rotating projection device is more complex, and its response time will be lower than that of a system with a projector rigidly mounted on a mobile apparatus 1.In a particular case, the system for detecting and determining the position of objects near the mobile apparatus 1 may employ several projection devices that form LM structures with different characteristics, such as periods and / or directions of the LM structure stripes, distances to the planes of greatest contrast of the LM structures, and / or their spatial orientation. A system for detecting and determining the position of objects near the mobile apparatus that utilizes several projection devices allows, in particular, for increasing the size of the monitored spatial region without degrading the system's performance. Figure 7 shows a diagram of the mobile apparatus 1 and a system installed thereon with two projectors 2 and 20 and two image recording devices 3 and 19, which allows for the detection and determination of the position of objects in front of and behind the mobile apparatus.Projection devices 2 and 20 form LM structures in front of and behind the mobile apparatus 1, respectively, and image recording devices 3 and 19 record images of the space in front of and behind the mobile apparatus 1, respectively. Detection and determination of the position of objects behind the mobile apparatus 1 is necessary, in particular, when it is moving backward, for example, when parking or reversing out of a parking space. In another particular case, the system for detecting and determining the position of objects may use two or more projection devices whose optical axes are directed at an angle to each other that is less than 180 degrees, for example, at an acute angle to each other (see Fig. 8). This makes it possible to increase the width of the spatial region that is monitored by the system, for example, in front of the mobile apparatus and to the side of it.To record images of the region of space illuminated by projection devices 2 and 20, one or more image recording devices may be used. Fig. 7 and Fig. 8 show image recording devices 3 and 19, whose optical axes are parallel to the optical axes of projection devices 2 and 20, respectively. In Fig. 8, the projection of these optical axes onto the y ^ const plane is shown by dashed lines. If the objects and obstacles that are likely to be encountered in the path of the moving apparatus are known, then in order to increase the reliability and accuracy of the method, as well as to reduce the time for measuring the position of such objects, it is advisable to optimize the characteristics of the LM structure formed by the projection device, in particular, to select the optimal values of the period and depth of field of the LM structure, as well as the direction of its stripes.For example, to detect and determine the position of relatively thin, approximately vertical objects whose width is comparable to the period of the LM structure or smaller than it, such as pipes, fence posts and gratings, lamp posts, road sign posts, power line supports, roofs, racks, tree trunks, etc., it is preferable to use in a system installed on a mobile apparatus at least one projector that forms the LM structure with a horizontally directed stripe. Fig. 9 schematically shows a system for detecting and determining the position of objects, in which two projectors with a horizontally directed stripe of the output grating are used. In a particular case of the implementation of the system (see Fig. 9), projector 2 is installed closer to the right edge of the mobile apparatus 1, and projector 20 - to the left, and one image recording device 3 is used in it, located on a line passing in the middle between projectors 2 and 20.The optical axes of projectors 2 and 20 and image recording devices 3 are parallel to the direction of rectilinear motion of mobile apparatus 1. More than one image recording device may be used in the system. Projectors 2 and 20 illuminate the spatial region in front of mobile apparatus 1. The light patterns projected by them onto objects 8 have horizontal stripes. In another particular embodiment of the method for detecting and determining the position of objects near a mobile apparatus, observation of the spatial region illuminated by at least one projection device 2 is performed by a driver or operator, using, for example, a television camera and monitor, or the naked eye.When observing at least one area 9 on the surface of any object 8 or on its image on the monitor screen, within which a quasi-periodic change in surface brightness with a known spatial period and a contrast of light and shadow bands clearly visible to the eye is visually registered, the driver or operator records the fact that the mobile apparatus 1 is approaching the object 8 or a part thereof at a known distance. Based on this information, he can decide to stop or change the direction of movement of the mobile apparatus 1. SOURCES OF INFORMATION 1. Perez L., Rodriguez I., Rodriguez N., Usamentiaga R., and Garcia DF "Robot Guidance Using Machine Vision Techniques in Industrial Environments: A Comparative Review" / / Sensors, 2016, Vol.16, No.3, pp.335-1 - 335-26; 2. Patent US 20080199069, 08 / 21 / 2008 “Stereo Camera for a Motor Vehicle” / / Schick J., Schmack A., Steffen F.; 3. Patent US 20160311443, 10.27.2016 «Stereo camera, vehicle driving auxiliary device having same, and vehicle» / / Hong Y., Park W., Park Y., Lee N.; 4. Patent US20160328619, 10.11.2016 «Vehicle driving assistance device and vehicle having the same» / / Yi C., Park Y., Sanjeevu M., Vijay R., Naresh R. Y.; 5. Patent US 8064285, 22.11.2011 «Method for functionally testing an ultrasonic sensor» / / Preissler P.; 6. Patent US 8942065, 27.01.2015 «Method and device for determining the position of an object in relation to a vehicle» / / Schmid D., Schumann M.; 7. Patent US 9557168, 31.01.2017 «Ultrasonic measuring system having a reduced minimum range and method for detecting an obstacle» / / Klotz A., Schmid D., Schumann M., Karl M.; 8. Patent US 10011278, 03.07.2018 «Collision avoidance system» / / Himi T.; 9. Patent US 10852130, 01.12.2020 «Proximity sensing systems and methods» / / Zhou W., Huang L.; 10. Patent US 8368873, 05.02.2013 «Proximity to target detection system and method» / / Oron M., Palatnik V.; 11.Patent US 8862271, 14.10.2014 «Proximity sensing on mobile robots» / / Shamilian S. V., Duffley S., Romanov N., Hook F. D., Munich M. E., Goel D.; 12. Patent US 11084172, 10.08.2021 «Obstacle following sensor scheme for a mobile robot» / / Casey C. M., Cross M., Ozick D. N., Jones J. L.; 13. Патент RU 2515312, 10.05.2014 «Оптический датчик для автомобиля» / / Шепеленко В. Б.; 14. Patent US 8332134, 11.12.2012 «Three-dimensional LIDAR-based clear path detection» / / Zhang W., Peterson K.; 15. Patent US 9329269, 03.05.2016 «Method for registration of range images from multiple LiDARS» / / Zeng S.; 16. Patent US 10447973, 15.10.2019 «Rotating LIDAR with co-aligned imager» / / Droz P.-Y., Verghese S., Hermalyn B.; 17. Patent US 10578719, 03.03.2020 «Vehicle-integrated LIDAR system» / / O'Keeffe J. T.; 18. Patent US 11442168, 13.09.2022 «Vehicular driving assist system with lidar sensors that emit different patterns of light» / / Nicke P., May B.; 19. Patent US 4954962, 04.09.1990 «Visual navigation and obstacle avoidance structured light system» / / Evans J. M., Jr., Weiman C. F. R., King S. J.; 20. Patent US 6204754, 20.03.2001 «Proximity indicating system» / / Berstis V.; 21. Patent US 20050073397, 07.04.2005 «Rear and front clearance light system for a vehicle» / / Zheng B., Chen Y.; 22. Patent US 20070177011, 02.08.2007 «Movement control system» / / Lewin A., Woods S., Orchard D.; 23. Patent US 20090309710, 17.12.2009 «Vehicle Vicinity Monitoring System» / / Kakinami T.; 24. Patent US 20140152823, 05.06.2014 «Techniques to Obtain Information About Objects Around a Vehicle» / / Breed D. S.; 25. Patent US 20170106793, 20.04.2017 «Vehicle lighting system with dynamic beam pattern» / / Kumar A., Michaels R. J. III, Bilger D. P., Golden E. R.; 26. Patent US 10719723, 21.07.2020 «Path sensing using structured lighting» / / Schamp G. G.; 27. Patent US 10807230, 20.10.2020 «Bistatic object detection apparatus and methods» / / Szatmary B., Richert M.; 28.Patent US 20220018964, 20.01.2022 «Surroundings detection device for vehicle» / / Horiba K., Kinoshita M., Saito H.; 29. Patent US 20230252660, 10.08.2023 «Image-capturing apparatus for vehicle» / / Takori M., Tsuchiya Y.; 30. Patent DE 102011017707, 31.10.2012 «Method for detecting object e.g. concrete post in vicinity of camera for monitoring rear side region of vehicle, involves reading images captured by cameras after partially illuminating with illumination patterns» / / Niehsen W., Simon S.; 31. Patent DE 102012207716, 14.11.2013 «Optical scanning system for scanning environment of motor car, has processing unit making combined evaluation of information of images, and illumination device provided in infrared region for illuminating environment» / / Focke T., Meschenmoser R., Engelberg T.; 32. Patent DE 102010039092, 22.10.2020 «Method and control device for determining a distance between an object and a vehicle» / / Roelke V.; 33. Patent DE 102014100579, 23.07.2015 "System for generating information about a projection surface in front of a motor vehicle" / / Kubitzka B., Wilks C.; 34. Guzhov V. I. Methods for measuring 3D profile of objects. Contact, triangulation systems and structured lighting methods, Novosibirsk, NSTU Publishing House, 2015, 82 p.; 35. Patent RU 2807409, 11 / 14 / 2023 "Method and system for contactless ranging and profilometry" / / Gitlin M. S.; 36. McCurry RE "Multiple Source Moiré Patterns with Photographic Diffraction Gratings" / / J. Appl. Phys., 1966, Vol.37, Issue 2, pp.479-482; 37. Keren E., Livnat A., and Glatt I. “Moire deflectometry with pure sinusoidal gratings” / / Optics Letters, 1985, Vol.10, No.4, pp.167-169; 38. Patorski K. “The Self-Imaging Phenomenon and Its Applications” / / Progress in Optics XXVII, Edited By E. Wolf, Elsevier Science Publishers, 1989, 107 p.; 39. Crespo D., Alonso J., and Bernabéu E. “Generalized imaging using an extended monochromatic light source,” J. Opt. Soc. Am. A, 2000, Vol.17, No.7, pp.1231-1240; 40. Iwata K., Sando Y., Satoh K., Moriwaki K. «Application of generalized grating imaging to pattern projection in three-dimensional profilometry» / / Appl. Opt., 2011, Vol.50, No.26, pp.5115-5121; 41. Hassani K., Nahal A., Tirandazi N. «Surface profilometry using the incoherent self- imaging technique in reflection mode» / / J. Appl. Phys., 2018, Vol.123, pp.035302-1-035302-6; 42. Toto-Arellano N.-I., Goméz J.M. M., García-Lechuga L., Montes-Peréz A., Rodríguez Zurita G., Martínez García A., Martínez Dominguéz J.A. «Diffraction theory of binary amplitude and phase gratings with applications for Ronchi test» / / Optik, 2015, Vol.126, Issue 23, pp.3717-3727; 43. Edmund optics, www.edmundoptics.eu 44. Applied Image Inc., www.appliedimage.com 45. Thorlabs, Inc. https: / / www.thorlabs.com 46. Buchwald K. «White paper: Fused Silica Transmission Gratings» / / Ibsen Photonics, 2007, Publication Version: 1.0 March 2007, https: / / ibsen.com / wp-content / uploads / White- paper-Fused-Silica-Transmission-Gratings-v1.0-1.pdf; 47. Patent JP 2013114025A, 29.11.2011 «Grating of Talbot-Lau interferometer» / / Iwata K.; 48. Iwata K., Satoh K., and Moriwaki K. «Relaxation of the Talbot condition in generalized grating imaging» / / Applied Optics, 2012, Vol.51, No.16, pp.3137-3144; 49. Patent US 5067816, 26.11.1991 «Optical encoder» / / Ichikawa S.; 50. Patent US 5155355, 13.10.1992 «Photoelectric encoder having a grating substrate with integral light emitting elements» / / Kabaya Y.; 51. Patent US 5576537, 19.11.1996 «Photoelectric position measuring system using gratings having specified dimensions to suppress harmonics» / / Holzapfel W., Franz A.; 52. Patent US 2016138945, 19.05.2016 «Illumination portion for an optical encoder» / / MacCoy B. K. H., Emtman C. E.; 53. Patent US 2016341576, 24.11.2016 «Light source array used in an illumination portion of an optical encoder» / / MacCoy B. K. H., Emtman C. E., Kapner D. J.; 54. Гужов В. И.Methods for measuring 3D profile of objects. Phase methods, Novosibirsk, NSTU Publishing House, 2016, 82 p.; 55. Gruzman I. S., Kirichuk V. S., Kosykh V. P., Peretyagin G. I., Spektor A. A. Digital image processing in information systems, Novosibirsk: NSTU Publishing House, 2000, 168 p.; 56. McCurry R. E. "Multiple Source Moiré Patterns" / / J. Appl. Phys., 1966, Vol. 37, Issue 2, pp. 467-472; 57. Swanson G. J. and Leith E. N. "Analysis of the Lau effect and generalized grating imaging" / / J. Opt. Soc. Am. A, 1985, Vol. 2, No. 6, pp. 789-793; 58. Iwata K. “Interpretation of generalized grating imaging” / / J. Opt. Soc. Am. A, 2008, Vol.25, No.9, pp.2244-2250; 59. Patent JP 2008232643 A, 10 / 02 / 2008 “Apparatus and method for measuring solid form” / / Tomii T., Kusunoki F., Iwata K.; Moriwaki K., Fukuda H.; 60. García-Rodríguez L., Alonso J. and Bernabéu E. “Grating pseudo-imaging with polychromatic and finite extension sources” / / Optics Express, 2004, Vol.12, No.11, pp.2529-2541; 61. Sanchez-Brea LM, Saez-Landete J., Alonso J., Bernabeu E. «Invariant grating pseudoimaging using polychromatic light and a finite extension source» / / Applied Optics, 2008, Vol.47, No.10, pp.1470-1477; 62. Ye G., Liu H., Fan S., Li X., Yu H., Lei B., Shi Y., Yin L., Lu B. «A theoretical investigation of generalized grating imaging and its application to optical encoder» / / Optics Communications, 2015, Vol.354, pp.21-27; 63. Dubetsky B. and Berman P. R. «Creating and probing subwavelength atomic gratings using spatially separated fields» / / Physical Review A, 1994, Vol.50, No.5, pp.4057- 4068; 64. Dubetsky B. and Berman P. R. «Atom interference using microfabricated structures» / / Atom Interferometry, Edited By Paul R. Berman, Academic Press, 1997, pp.407-468; 65. Iwata K. «Interpretation of generalized grating imaging: further analysis and numerical calculation» / / J. Opt. Soc. Am. A, 2008, Vol.25, No.12, pp.2939-2944; 66. Case W. B., Tomandl M., Deachapunya S., and Arndt M."Implementation of optical carpets in the Talbot and Talbot-Lau configurations" / / Opt. Express, 2009, Vol.17, Issue 23, pp.20966-20974; 67. Lider V.V. "Talbot and Talbot-Lau X-ray interferometers" / / Phys. Usp., 2023, Vol.193, No.10, pp.1047-1070; 68. Zhang S. High-Speed 3D Imaging with Digital Fringe Projection Techniques, CRC Press, 2019, 216 p.; 69. Patent US 20150192676, 07 / 09 / 2015 “LiDAR System Comprising A Single-Photon Detector” / / Kotelnikov E. Yu., Mordarski WP, Kudryashov I., Rangwala SS
Claims
Claim 1. A system for detecting and determining the position of objects near a mobile apparatus, including at least one projection device emitting structured light, which is mounted on the mobile apparatus, characterized in that a wide-field projection device is used, containing an illuminator and at least one flat periodic transmitting grating, which is illuminated by the illuminator, wherein the dimensions of the grating, as well as the dimensions of the working aperture of the illuminator and the width of the directional pattern of the light emitted by it are such that they allow the projection device to form at least one light localized moiré structure with a depth of field that is sufficient to detect and determine the position of the suspected objects.
2. The system according to claim1, characterized in that it uses at least one wide-field projection device, the dimensions of which are the grating, as well as the dimensions of the working aperture of the illuminator and the width of the directional pattern of the light emitted by it, allow the projection device to form at least one light localized moire structure with a depth of field of less than 10%.
3. The system according to claim 1, characterized in that it additionally uses at least one image recording device, which is mounted on a movable apparatus and which performs the recording of images of at least part of the region of space illuminated by the projection device, forming at least one light localized moire structure.
4. The system according to claim3, characterized in that it additionally uses at least one computing device that can exchange data and control the image recording device and other devices of the system, as well as control devices or devices for assisting in the control of the mobile apparatus.
5. The system according to claim 1, including at least one projection device containing two flat one-dimensional periodic transmission gratings that are located in parallel planes on one side of the illuminator and have the same direction of stripes and that are illuminated by this illuminator, wherein the light that has passed through the first grating falls on the second grating, which is the output grating, characterized in that the dimensions of the gratings, as well as the dimensions of the working aperture.
6. The system of claim 5, wherein the projection device uses an illuminator with a uniform flat luminous surface that is parallel to the planes of the gratings, and the width of the working aperture of the illuminator and the width of the directional pattern of the light emitted by it are such that they allow the formation of at least one localized moire light structure with a depth of field that is sufficient to detect and determine the position of the intended objects.
7. The system of claim 5, wherein a diffuse light source is used as the illuminator in the optical projection device.
8. The system of claim5, characterized in that it uses a projection device in which a device is installed that controllably changes the distance between the planes of the first and second gratings.
9. The system according to claim 1, characterized in that it uses a projection device in which the characteristics of at least one grating can be changed.
10. The system according to claim 1, characterized in that it uses a projection device in which a spatial light modulator is used to form at least one grating.
11. The system according to claim1, characterized in that it uses a projection device in which one flat one-dimensional periodic transmitting grating is installed, which is the output grating, and an illuminator is used, which is a one-dimensional periodic grating of light sources, the light from which falls on the output grating, wherein the luminous surfaces of the individual light sources are located in a plane parallel to the plane of the output grating, and their luminous stripes are directed parallel to the stripes of the output grating.
12. The system according to claim 1, characterized in that it uses a projection device in which one flat one-dimensional periodic transmitting grating is installed, which is the output grating, and an illuminator is used, which is a periodic line of light sources, the light from which falls on the output grating, wherein the centers of the luminous surfaces of the individual light sources.
13. The system of claim 1, characterized in that it uses a projection device in which one flat one-dimensional periodic transmission grating is installed, which is the output grating, and an illuminator is used, which is a rectangular periodic matrix of light sources, the light from which falls on the output grating, wherein the luminous surfaces of the individual light sources of such an illuminator are located in a plane parallel to the plane of the output grating, and the columns or rows of the matrix of light sources are parallel to the direction of the stripes of the output grating.
14. The system of any one of claims 11-13, characterized in that it uses a projection device in which a device is installed that controllably changes the distance between the plane of the output grating and the illuminator.
15. The system of any one of claims5, 11-13, characterized in that it uses a projection device, in which a device is additionally installed that controllably moves at least one grating along a transverse coordinate perpendicular to the direction of its stripes.
16. The system according to any of paragraphs. 11-13, characterized in that it uses a projection device, in which a device is additionally installed that controllably moves the illuminator along a transverse coordinate perpendicular to the direction of the stripes of the output grating.
17. The system according to claim 1, characterized in that it uses a projection device that can operate in a pulsed or pulse-periodic mode.
18. The system according to claim 1, characterized in that it uses a projection device that can rotate relative to the movable apparatus.
19. The system according to claim 1, characterized in that it uses two or more projection devices, the direction of the optical axes of which do not coincide with each other. 20.A method for detecting and determining the position of objects near a mobile apparatus, which includes illuminating a region of space near the mobile apparatus with structured light emitted by at least one projection device that is installed on the mobile apparatus, characterized in that structured light is used to illuminate the region of space. represents at least one localized moiré structure, the fact of approaching the mobile apparatus to the object or to its part at a known distance is recorded by registering at least one area on the surface of the object, within which a quasi-periodic change in brightness with a known spatial period is observed, wherein the distance is determined by the contrast of the projection of the localized moiré structure with a known depth of field.
21. The method according to claim 20, characterized in that for its implementation at least one localized moiré structure is used, which has the highest contrast value of more than 0.
2.
22. The method according to claim20, characterized in that the registration of images of the region of space illuminated by the projection device is carried out using at least one image registration device, the dependence of the brightness of these images on 2D coordinates in the images is calculated by a computing device, the dependence of the contrast of the quasi-periodic change in brightness with a known spatial period on 2D coordinates in the images is calculated, the fact of approaching the object or part thereof to the plane of the output grating, which is installed in the projection device, at a known distance is recorded if there is at least one region in the image within which the contrast of the quasi-periodic change in brightness of the image with the said spatial period exceeds a specified value.
23. The method according to claim22, characterized in that the coordinates of the region of the object within which the contrast of the quasi-periodic change in the brightness of the image with a known spatial period exceeds a given value are determined in the coordinate system associated with the projection device, taking into account that the plane where the contrast value of the localized moire structure is constant is parallel to the plane of the output grating of the projector and is removed from it at a known distance, and also, taking into account the results of the calibration of the image recording device, the position of the said region of the surface of the object relative to the moving apparatus and its parts is determined, using for this purpose information about the shape of the moving apparatus and the location of the projection device on it.
24. The method according to claim 20, characterized in that the observation of the region of space that is illuminated by the projection device and the detection of objects is carried out.
25. The method according to claim 20, characterized in that the detection of objects is carried out by the operator controlling the mobile apparatus by recording on the screen of a monitor displaying information from the image recording device an image of at least one area on the surface of an object, within which a quasi-periodic change in its brightness with a known spatial period is observed.
Citation Information
Patent Citations
Moire device
JP2000009445A
Apparatus and method for measuring solid form
JP2008232643A
Technique optically measuring form of surface of three-dimensional object (variants)
RU2232373C1
Techniques to Obtain Information About Objects Around a Vehicle
US20140152823A1
Surroundings detection device for vehicle
US20220018964A1